{ "cells": [ { "cell_type": "markdown", "id": "2f04eee0-5928-4e74-a754-6dc2e528810c", "metadata": {}, "source": [ "# GrimAge2PackYrs" ] }, { "cell_type": "markdown", "id": "a3f514a3-772c-4a14-afdf-5a8376851ff4", "metadata": {}, "source": [ "## Index\n", "1. [Instantiate model class](#Instantiate-model-class)\n", "2. [Define clock metadata](#Define-clock-metadata)\n", "3. [Download clock dependencies](#Download-clock-dependencies)\n", "5. [Load features](#Load-features)\n", "6. [Load weights into base model](#Load-weights-into-base-model)\n", "7. [Load reference values](#Load-reference-values)\n", "8. [Load preprocess and postprocess objects](#Load-preprocess-and-postprocess-objects)\n", "10. [Check all clock parameters](#Check-all-clock-parameters)\n", "10. [Basic test](#Basic-test)\n", "11. [Save torch model](#Save-torch-model)\n", "12. [Clear directory](#Clear-directory)" ] }, { "cell_type": "markdown", "id": "d95fafdc-643a-40ea-a689-200bd132e90c", "metadata": {}, "source": [ "Let's first import some packages:" ] }, { "cell_type": "code", "execution_count": 18, "id": "4adfb4de-cd79-4913-a1af-9e23e9e236c9", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:41.316681Z", "iopub.status.busy": "2025-04-07T17:51:41.316440Z", "iopub.status.idle": "2025-04-07T17:51:42.738147Z", "shell.execute_reply": "2025-04-07T17:51:42.737780Z" } }, "outputs": [], "source": [ "import os\n", "import inspect\n", "import shutil\n", "import json\n", "import torch\n", "import pandas as pd\n", "import pyaging as pya\n", "import numpy as np" ] }, { "cell_type": "markdown", "id": "145082e5-ced4-47ae-88c0-cb69773e3c5a", "metadata": {}, "source": [ "## Instantiate model class" ] }, { "cell_type": "code", "execution_count": 19, "id": "8aa77372-7ed3-4da7-abc9-d30372106139", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:42.740018Z", "iopub.status.busy": "2025-04-07T17:51:42.739761Z", "iopub.status.idle": "2025-04-07T17:51:42.750935Z", "shell.execute_reply": "2025-04-07T17:51:42.750574Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "class GrimAge2PackYrs(pyagingModel):\n", " def __init__(self):\n", " super().__init__()\n", "\n", " def preprocess(self, x):\n", " return x\n", "\n", " def postprocess(self, x):\n", " return x\n", "\n" ] } ], "source": [ "def print_entire_class(cls):\n", " source = inspect.getsource(cls)\n", " print(source)\n", "\n", "print_entire_class(pya.models.GrimAge2PackYrs)" ] }, { "cell_type": "code", "execution_count": 20, "id": "78536494-f1d9-44de-8583-c89a310d2307", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:42.753015Z", "iopub.status.busy": "2025-04-07T17:51:42.752877Z", "iopub.status.idle": "2025-04-07T17:51:42.754599Z", "shell.execute_reply": "2025-04-07T17:51:42.754314Z" } }, "outputs": [], "source": [ "model = pya.models.GrimAge2PackYrs()" ] }, { "cell_type": "markdown", "id": "51f8615e-01fa-4aa5-b196-3ee2b35d261c", "metadata": {}, "source": [ "## Define clock metadata" ] }, { "cell_type": "code", "execution_count": 21, "id": "135ce001-03f7-4025-bceb-01a3e2e2b0ef", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:42.755964Z", "iopub.status.busy": "2025-04-07T17:51:42.755873Z", "iopub.status.idle": "2025-04-07T17:51:42.758035Z", "shell.execute_reply": "2025-04-07T17:51:42.757775Z" } }, "outputs": [], "source": [ "model.metadata[\"clock_name\"] = \"grimage2packyrs\"\n", "model.metadata[\"data_type\"] = \"DNA methylation\" # Paper: Blood DNA methylation profiles were measured to construct the biomarkers.\n", "model.metadata[\"species\"] = \"Homo sapiens\" # Paper: The training set comprised human FHS Offspring participants.\n", "model.metadata[\"year\"] = 2022\n", "model.metadata[\"approved_by_author\"] = \"⌛\"\n", "model.metadata[\"citation\"] = \"Lu, A. T., et al. “DNA methylation GrimAge version 2.” Aging 14(23): 9484–9549 (2022).\"\n", "model.metadata[\"doi\"] = \"https://doi.org/10.18632/aging.204434\"\n", "model.metadata[\"notes\"] = \"DNAm PACKYRS is a methylation surrogate for cumulative smoking exposure and a GrimAge component.\"\n", "model.metadata[\"research_only\"] = True\n", "model.metadata[\"tissue\"] = [\"whole blood\"] # Paper: Peripheral blood/buffy-coat DNA from FHS exam 8 supplied the methylation predictors.\n", "model.metadata[\"predicts\"] = [\"smoking exposure\"] # Paper: DNAm smoking pack-years is a DNAm-based surrogate of the corresponding observed biomarker.\n", "model.metadata[\"training_target\"] = [\"smoking exposure\"] # Paper: Smoking Pack-Years was one of the observed plasma-protein or smoking variables estimated by a DNAm surrogate.\n", "model.metadata[\"unit\"] = [\"pack-years\"] # Paper: Smoking pack-years equals packs per day multiplied by years smoked.\n", "model.metadata[\"model_type\"] = \"elastic net regression\" # Paper: Stage-1 DNAm surrogates were fit by elastic-net regression with ten-fold cross-validation.\n", "model.metadata[\"platform\"] = [\"Illumina 450K\"] # Paper: Training methylation profiling was based on the Illumina HumanMethylation450K BeadChip.\n", "model.metadata[\"population\"] = \"older adults\" # Paper: GrimAge2 was trained in 1,833 FHS participants aged 40–92 years.\n", "model.metadata[\"journal\"] = \"Aging\"\n", "model.metadata[\"last_author\"] = \"Steve Horvath\"\n", "model.metadata[\"n_features\"] = 173\n", "model.metadata[\"citations\"] = 291\n", "model.metadata[\"citations_date\"] = \"2026-07-05\"\n" ] }, { "cell_type": "markdown", "id": "74492239-5aae-4026-9d90-6bc9c574c110", "metadata": {}, "source": [ "## Download clock dependencies" ] }, { "cell_type": "code", "execution_count": 22, "id": "95f6ba57", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "|-----------> Downloading data to ./grimage2_subcomponents.csv\n", "|-----------> in progress: 100.0000%\n", "|-----------> Downloading data to ./grimage2.csv\n", "|-----------> in progress: 100.0000%%\n", "|-----------> Downloading data to ./datMiniAnnotation3_Gold.csv\n", "|-----------> in progress: 100.0000%\n" ] } ], "source": [ "logger = pya.logger.Logger()\n", "urls = [\n", " \"https://huggingface.co/lucascamillomd/pyaging-data/resolve/main/supporting_files/grimage2_subcomponents.csv\",\n", " \"https://huggingface.co/lucascamillomd/pyaging-data/resolve/main/supporting_files/grimage2.csv\",\n", " \"https://huggingface.co/lucascamillomd/pyaging-data/resolve/main/supporting_files/datMiniAnnotation3_Gold.csv\",\n", "]\n", "dir = \".\"\n", "for url in urls:\n", " pya.utils.download(url, dir, logger, indent_level=1)" ] }, { "cell_type": "markdown", "id": "a14c7fc1-abe5-42a3-8bc9-0987521ddf33", "metadata": {}, "source": [ "## Load features" ] }, { "cell_type": "markdown", "id": "3e737582-3a28-4f55-8da9-3e34125362cc", "metadata": {}, "source": [ "#### From CSV" ] }, { "cell_type": "code", "execution_count": 23, "id": "f1486db4", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array(['DNAmGDF_15', 'DNAmB2M', 'DNAmCystatin_C', 'DNAmTIMP_1', 'DNAmadm',\n", " 'DNAmpai_1', 'DNAmleptin', 'DNAmPACKYRS', 'DNAmlog.CRP',\n", " 'DNAmlog.A1C'], dtype=object)" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df = pd.read_csv('grimage2_subcomponents.csv', index_col=0)\n", "df['Y.pred'].unique()" ] }, { "cell_type": "code", "execution_count": 24, "id": "f7ad18f4", "metadata": {}, "outputs": [], "source": [ "df = df[df['Y.pred'] == 'DNAmPACKYRS']\n", "df['feature'] = df['var']\n", "df['coefficient'] = df['beta']\n", "model.features = ['age'] + df['feature'][2:].tolist()" ] }, { "cell_type": "code", "execution_count": 25, "id": "ad412f55", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Y.predvarbetafeaturecoefficient
1622DNAmPACKYRSIntercept-31.997022Intercept-31.997022
1623DNAmPACKYRSAge0.142144Age0.142144
1624DNAmPACKYRScg2046244914.697949cg2046244914.697949
1625DNAmPACKYRScg198023900.459989cg198023900.459989
1626DNAmPACKYRScg140849070.382296cg140849070.382296
\n", "
" ], "text/plain": [ " Y.pred var beta feature coefficient\n", "1622 DNAmPACKYRS Intercept -31.997022 Intercept -31.997022\n", "1623 DNAmPACKYRS Age 0.142144 Age 0.142144\n", "1624 DNAmPACKYRS cg20462449 14.697949 cg20462449 14.697949\n", "1625 DNAmPACKYRS cg19802390 0.459989 cg19802390 0.459989\n", "1626 DNAmPACKYRS cg14084907 0.382296 cg14084907 0.382296" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.head()" ] }, { "cell_type": "markdown", "id": "ee6d8fa0-4767-4c45-9717-eb1c95e2ddc0", "metadata": {}, "source": [ "## Load weights into base model" ] }, { "cell_type": "markdown", "id": "3958ba73-42e8-40a5-94a1-4f4b8ae05dca", "metadata": {}, "source": [ "#### Linear model" ] }, { "cell_type": "code", "execution_count": 26, "id": "321a437c-8888-4e10-96e9-5ed2826a8f74", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.647294Z", "iopub.status.busy": "2025-04-07T17:51:44.647116Z", "iopub.status.idle": "2025-04-07T17:51:44.688112Z", "shell.execute_reply": "2025-04-07T17:51:44.687757Z" } }, "outputs": [], "source": [ "weights = torch.tensor(df['coefficient'][1:].tolist()).unsqueeze(0)\n", "intercept = torch.tensor([df['coefficient'].iloc[0]])" ] }, { "cell_type": "markdown", "id": "5742dc16-e063-414f-a38e-9721beb11351", "metadata": {}, "source": [ "#### Linear model" ] }, { "cell_type": "code", "execution_count": 27, "id": "c2e54115-c17b-48ce-88f1-de546c90d2b3", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.689783Z", "iopub.status.busy": "2025-04-07T17:51:44.689683Z", "iopub.status.idle": "2025-04-07T17:51:44.692010Z", "shell.execute_reply": "2025-04-07T17:51:44.691738Z" } }, "outputs": [], "source": [ "base_model = pya.models.LinearModel(input_dim=len(model.features))\n", "\n", "base_model.linear.weight.data = weights.float()\n", "base_model.linear.bias.data = intercept.float()\n", "\n", "model.base_model = base_model" ] }, { "cell_type": "markdown", "id": "ad8b4c1d-9d57-48b7-9a30-bcfea7b747b1", "metadata": {}, "source": [ "## Load reference values" ] }, { "cell_type": "code", "execution_count": 28, "id": "2089b66f-9cc4-4528-9bdc-5e45efc6d06b", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.693409Z", "iopub.status.busy": "2025-04-07T17:51:44.693309Z", "iopub.status.idle": "2025-04-07T17:51:44.708180Z", "shell.execute_reply": "2025-04-07T17:51:44.707878Z" } }, "outputs": [], "source": [ "reference_df = pd.read_csv('datMiniAnnotation3_Gold.csv', index_col=0)\n", "model.reference_values = [65] + reference_df.loc[model.features[1:]]['gold'].tolist()" ] }, { "cell_type": "markdown", "id": "af3bcf7b-74a8-4d21-9ccb-4de0c2b0516b", "metadata": {}, "source": [ "## Load preprocess and postprocess objects" ] }, { "cell_type": "code", "execution_count": 29, "id": "7a22fb20-c605-424d-8efb-7620c2c0755c", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.709686Z", "iopub.status.busy": "2025-04-07T17:51:44.709594Z", "iopub.status.idle": "2025-04-07T17:51:44.711225Z", "shell.execute_reply": "2025-04-07T17:51:44.710973Z" } }, "outputs": [], "source": [ "model.preprocess_name = None\n", "model.preprocess_dependencies = None" ] }, { "cell_type": "code", "execution_count": 30, "id": "ff4a21cb-cf41-44dc-9ed1-95cf8aa15772", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.712430Z", "iopub.status.busy": "2025-04-07T17:51:44.712349Z", "iopub.status.idle": "2025-04-07T17:51:44.713842Z", "shell.execute_reply": "2025-04-07T17:51:44.713585Z" } }, "outputs": [], "source": [ "model.postprocess_name = None\n", "model.postprocess_dependencies = None" ] }, { "cell_type": "markdown", "id": "86e3d6b1-e67e-4f3d-bd39-0ebec5726c3c", "metadata": {}, "source": [ "## Check all clock parameters" ] }, { "cell_type": "code", "execution_count": 31, "id": "2168355c-47d9-475d-b816-49f65e74887c", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.715112Z", "iopub.status.busy": "2025-04-07T17:51:44.715032Z", "iopub.status.idle": "2025-04-07T17:51:44.726874Z", "shell.execute_reply": "2025-04-07T17:51:44.726577Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "%==================================== Model Details ====================================%\n", "Model Attributes:\n", "\n", "training: True\n", "metadata: {'approved_by_author': '⌛',\n", " 'citation': 'Lu, Ake T., et al. \"DNA methylation GrimAge version 2.\" Aging '\n", " '(Albany NY) 14.23 (2022): 9484.',\n", " 'clock_name': 'grimage2packyrs',\n", " 'data_type': 'methylation',\n", " 'doi': 'https://doi.org/10.18632/aging.204434',\n", " 'notes': None,\n", " 'research_only': True,\n", " 'species': 'Homo sapiens',\n", " 'version': None,\n", " 'year': 2022}\n", "reference_values: [65, 0.936110502255403, 0.46754473577805, 0.101838529204782, 0.668699052138661, 0.408180706152194, 0.033798963669035, 0.044827788203083, 0.91516014793103, 0.855444378968262, 0.189367366387564, 0.949642624900366, 0.430182534715076, 0.550120185761743, 0.382373523551351, 0.151011071218822, 0.750308469962453, 0.0481653215294158, 0.0623779128286142, 0.795309058162839, 0.911175830780723, 0.0419140679935733, 0.0388065967727853, 0.854286856105627, 0.514045648234368, 0.935109546405548, 0.291483494927766, 0.064722337908201, 0.545425926051344, 0.0556280013236613]... [Total elements: 173]\n", "preprocess_name: None\n", "preprocess_dependencies: None\n", "postprocess_name: 'cox_to_years'\n", "postprocess_dependencies: None\n", "features: ['age', 'cg20462449', 'cg19802390', 'cg14084907', 'cg23251761', 'cg14825555', 'cg04967775', 'cg05817517', 'cg00500789', 'cg23800435', 'cg08871545', 'cg04135242', 'cg25189904', 'cg16511983', 'cg12067764', 'cg09396704', 'cg17440248', 'cg01491219', 'cg02356786', 'cg08980304', 'cg26881591', 'cg14528537', 'cg04053045', 'cg05515143', 'cg06625640', 'cg00102512', 'cg06372850', 'cg16978914', 'cg00706683', 'cg06644428']... [Total elements: 173]\n", "base_model_features: None\n", "\n", "%==================================== Model Details ====================================%\n", "Model Structure:\n", "\n", "base_model: LinearModel(\n", " (linear): Linear(in_features=173, out_features=1, bias=True)\n", ")\n", "\n", "%==================================== Model Details ====================================%\n", "Model Parameters and Weights:\n", "\n", "base_model.linear.weight: [0.14214389026165009, 14.697949409484863, 0.4599894881248474, 0.3822956085205078, 7.98643684387207, 1.6803100109100342, 1.0967497825622559, 16.303823471069336, 2.4014580249786377, 0.6859070062637329, 1.6773189306259155, 21.501564025878906, -2.096100330352783, 2.2927305698394775, 0.12879624962806702, 0.5189002752304077, 9.517245292663574, 1.3636956214904785, 1.7754020690917969, 2.1244921684265137, 3.7083091735839844, 3.0460753440856934, 1.3274203538894653, -0.6062915921211243, -1.1171971559524536, -13.956497192382812, 0.36579036712646484, -0.6485168933868408, 4.881432056427002, -24.69486427307129]... [Tensor of shape torch.Size([1, 173])]\n", "base_model.linear.bias: tensor([-31.9970])\n", "\n", "%==================================== Model Details ====================================%\n", "\n" ] } ], "source": [ "pya.utils.print_model_details(model)" ] }, { "cell_type": "markdown", "id": "986d0262-e0c7-4036-b687-dee53ba392fb", "metadata": {}, "source": [ "## Basic test" ] }, { "cell_type": "code", "execution_count": 32, "id": "936b9877-d076-4ced-99aa-e8d4c58c5caf", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.728239Z", "iopub.status.busy": "2025-04-07T17:51:44.728153Z", "iopub.status.idle": "2025-04-07T17:51:44.733560Z", "shell.execute_reply": "2025-04-07T17:51:44.733262Z" } }, "outputs": [ { "data": { "text/plain": [ "tensor([[ 17.0963],\n", " [ 118.3757],\n", " [-151.4175],\n", " [-116.8670],\n", " [ 181.9866],\n", " [ -9.3648],\n", " [ 91.3053],\n", " [-235.3991],\n", " [ -64.6384],\n", " [ 68.6570]], dtype=torch.float64, grad_fn=)" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch.manual_seed(42)\n", "input = torch.randn(10, len(model.features), dtype=float).double()\n", "model.eval()\n", "model.to(float)\n", "pred = model(input)\n", "pred" ] }, { "cell_type": "markdown", "id": "fe8299d7-9285-4e22-82fd-b664434b4369", "metadata": {}, "source": [ "## Save torch model" ] }, { "cell_type": "code", "execution_count": 33, "id": "5ef2fa8d-c80b-4fdd-8555-79c0d541788e", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.734916Z", "iopub.status.busy": "2025-04-07T17:51:44.734814Z", "iopub.status.idle": "2025-04-07T17:51:44.739278Z", "shell.execute_reply": "2025-04-07T17:51:44.738936Z" } }, "outputs": [], "source": [ "torch.save(model, f\"../weights/{model.metadata['clock_name']}.pt\")" ] }, { "cell_type": "markdown", "id": "bac6257b-8d08-4a90-8d0b-7f745dc11ac1", "metadata": {}, "source": [ "## Clear directory\n", "" ] }, { "cell_type": "code", "execution_count": 34, "id": "11aeaa70-44c0-42f9-86d7-740e3849a7a6", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.740582Z", "iopub.status.busy": "2025-04-07T17:51:44.740494Z", "iopub.status.idle": "2025-04-07T17:51:44.743819Z", "shell.execute_reply": "2025-04-07T17:51:44.743572Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Deleted file: grimage2_subcomponents.csv\n", "Deleted file: datMiniAnnotation3_Gold.csv\n", "Deleted file: grimage2.csv\n" ] } ], "source": [ "# Function to remove a folder and all its contents\n", "def remove_folder(path):\n", " try:\n", " shutil.rmtree(path)\n", " print(f\"Deleted folder: {path}\")\n", " except Exception as e:\n", " print(f\"Error deleting folder {path}: {e}\")\n", "\n", "# Get a list of all files and folders in the current directory\n", "all_items = os.listdir('.')\n", "\n", "# Loop through the items\n", "for item in all_items:\n", " # Check if it's a file and does not end with .ipynb\n", " if os.path.isfile(item) and not item.endswith('.ipynb'):\n", " os.remove(item)\n", " print(f\"Deleted file: {item}\")\n", " # Check if it's a folder\n", " elif os.path.isdir(item):\n", " remove_folder(item)" ] } ], "metadata": { "kernelspec": { "display_name": "research", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.7" } }, "nbformat": 4, "nbformat_minor": 5 }