{ "cells": [ { "cell_type": "markdown", "id": "2f04eee0-5928-4e74-a754-6dc2e528810c", "metadata": {}, "source": [ "# GrimAge2B2M" ] }, { "cell_type": "markdown", "id": "a3f514a3-772c-4a14-afdf-5a8376851ff4", "metadata": {}, "source": [ "## Index\n", "1. [Instantiate model class](#Instantiate-model-class)\n", "2. [Define clock metadata](#Define-clock-metadata)\n", "3. [Download clock dependencies](#Download-clock-dependencies)\n", "5. [Load features](#Load-features)\n", "6. [Load weights into base model](#Load-weights-into-base-model)\n", "7. [Load reference values](#Load-reference-values)\n", "8. [Load preprocess and postprocess objects](#Load-preprocess-and-postprocess-objects)\n", "10. [Check all clock parameters](#Check-all-clock-parameters)\n", "10. [Basic test](#Basic-test)\n", "11. [Save torch model](#Save-torch-model)\n", "12. [Clear directory](#Clear-directory)" ] }, { "cell_type": "markdown", "id": "d95fafdc-643a-40ea-a689-200bd132e90c", "metadata": {}, "source": [ "Let's first import some packages:" ] }, { "cell_type": "code", "execution_count": 17, "id": "4adfb4de-cd79-4913-a1af-9e23e9e236c9", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:41.316681Z", "iopub.status.busy": "2025-04-07T17:51:41.316440Z", "iopub.status.idle": "2025-04-07T17:51:42.738147Z", "shell.execute_reply": "2025-04-07T17:51:42.737780Z" } }, "outputs": [], "source": [ "import os\n", "import inspect\n", "import shutil\n", "import json\n", "import torch\n", "import pandas as pd\n", "import pyaging as pya\n", "import numpy as np" ] }, { "cell_type": "markdown", "id": "145082e5-ced4-47ae-88c0-cb69773e3c5a", "metadata": {}, "source": [ "## Instantiate model class" ] }, { "cell_type": "code", "execution_count": 18, "id": "8aa77372-7ed3-4da7-abc9-d30372106139", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:42.740018Z", "iopub.status.busy": "2025-04-07T17:51:42.739761Z", "iopub.status.idle": "2025-04-07T17:51:42.750935Z", "shell.execute_reply": "2025-04-07T17:51:42.750574Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "class GrimAge2B2M(pyagingModel):\n", " def __init__(self):\n", " super().__init__()\n", "\n", " def preprocess(self, x):\n", " return x\n", "\n", " def postprocess(self, x):\n", " return x\n", "\n" ] } ], "source": [ "def print_entire_class(cls):\n", " source = inspect.getsource(cls)\n", " print(source)\n", "\n", "print_entire_class(pya.models.GrimAge2B2M)" ] }, { "cell_type": "code", "execution_count": 19, "id": "78536494-f1d9-44de-8583-c89a310d2307", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:42.753015Z", "iopub.status.busy": "2025-04-07T17:51:42.752877Z", "iopub.status.idle": "2025-04-07T17:51:42.754599Z", "shell.execute_reply": "2025-04-07T17:51:42.754314Z" } }, "outputs": [], "source": [ "model = pya.models.GrimAge2B2M()" ] }, { "cell_type": "markdown", "id": "51f8615e-01fa-4aa5-b196-3ee2b35d261c", "metadata": {}, "source": [ "## Define clock metadata" ] }, { "cell_type": "code", "execution_count": 20, "id": "135ce001-03f7-4025-bceb-01a3e2e2b0ef", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:42.755964Z", "iopub.status.busy": "2025-04-07T17:51:42.755873Z", "iopub.status.idle": "2025-04-07T17:51:42.758035Z", "shell.execute_reply": "2025-04-07T17:51:42.757775Z" } }, "outputs": [], "source": [ "model.metadata[\"clock_name\"] = \"grimage2b2m\"\n", "model.metadata[\"data_type\"] = \"DNA methylation\" # Paper: Blood DNA methylation profiles were measured to construct the biomarkers.\n", "model.metadata[\"species\"] = \"Homo sapiens\" # Paper: The training set comprised human FHS Offspring participants.\n", "model.metadata[\"year\"] = 2022\n", "model.metadata[\"approved_by_author\"] = \"⌛\"\n", "model.metadata[\"citation\"] = \"Lu, A. T., et al. “DNA methylation GrimAge version 2.” Aging 14(23): 9484–9549 (2022).\"\n", "model.metadata[\"doi\"] = \"https://doi.org/10.18632/aging.204434\"\n", "model.metadata[\"notes\"] = \"B2M is linked to kidney function, cardiovascular disease and inflammation; DNAm B2M is a GrimAge component.\"\n", "model.metadata[\"research_only\"] = True\n", "model.metadata[\"tissue\"] = [\"whole blood\"] # Paper: Peripheral blood/buffy-coat DNA from FHS exam 8 supplied the methylation predictors.\n", "model.metadata[\"predicts\"] = [\"beta-2-microglobulin\"] # Paper: DNAm beta-2-microglobulin is a DNAm-based surrogate of the corresponding observed biomarker.\n", "model.metadata[\"training_target\"] = [\"beta-2-microglobulin\"] # Paper: Beta-2-Microglobulin was one of the observed plasma-protein or smoking variables estimated by a DNAm surrogate.\n", "model.metadata[\"unit\"] = [\"picograms per milliliter\"] # Paper: The inherited GrimAge plasma-protein targets were immunoassay measurements in pg/mL.\n", "model.metadata[\"model_type\"] = \"elastic net regression\" # Paper: Stage-1 DNAm surrogates were fit by elastic-net regression with ten-fold cross-validation.\n", "model.metadata[\"platform\"] = [\"Illumina 450K\"] # Paper: Training methylation profiling was based on the Illumina HumanMethylation450K BeadChip.\n", "model.metadata[\"population\"] = \"older adults\" # Paper: GrimAge2 was trained in 1,833 FHS participants aged 40–92 years.\n", "model.metadata[\"journal\"] = \"Aging\"\n", "model.metadata[\"last_author\"] = \"Steve Horvath\"\n", "model.metadata[\"n_features\"] = 92\n", "model.metadata[\"citations\"] = 291\n", "model.metadata[\"citations_date\"] = \"2026-07-05\"\n" ] }, { "cell_type": "markdown", "id": "74492239-5aae-4026-9d90-6bc9c574c110", "metadata": {}, "source": [ "## Download clock dependencies" ] }, { "cell_type": "code", "execution_count": 21, "id": "95f6ba57", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "|-----------> Downloading data to ./grimage2_subcomponents.csv\n", "|-----------> in progress: 100.0000%\n", "|-----------> Downloading data to ./grimage2.csv\n", "|-----------> in progress: 100.0000%%\n", "|-----------> Downloading data to ./datMiniAnnotation3_Gold.csv\n", "|-----------> in progress: 100.0000%\n" ] } ], "source": [ "logger = pya.logger.Logger()\n", "urls = [\n", " \"https://huggingface.co/lucascamillomd/pyaging-data/resolve/main/supporting_files/grimage2_subcomponents.csv\",\n", " \"https://huggingface.co/lucascamillomd/pyaging-data/resolve/main/supporting_files/grimage2.csv\",\n", " \"https://huggingface.co/lucascamillomd/pyaging-data/resolve/main/supporting_files/datMiniAnnotation3_Gold.csv\",\n", "]\n", "dir = \".\"\n", "for url in urls:\n", " pya.utils.download(url, dir, logger, indent_level=1)" ] }, { "cell_type": "markdown", "id": "a14c7fc1-abe5-42a3-8bc9-0987521ddf33", "metadata": {}, "source": [ "## Load features" ] }, { "cell_type": "markdown", "id": "3e737582-3a28-4f55-8da9-3e34125362cc", "metadata": {}, "source": [ "#### From CSV" ] }, { "cell_type": "code", "execution_count": 22, "id": "f1486db4", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array(['DNAmGDF_15', 'DNAmB2M', 'DNAmCystatin_C', 'DNAmTIMP_1', 'DNAmadm',\n", " 'DNAmpai_1', 'DNAmleptin', 'DNAmPACKYRS', 'DNAmlog.CRP',\n", " 'DNAmlog.A1C'], dtype=object)" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df = pd.read_csv('grimage2_subcomponents.csv', index_col=0)\n", "df['Y.pred'].unique()" ] }, { "cell_type": "code", "execution_count": 23, "id": "f7ad18f4", "metadata": {}, "outputs": [], "source": [ "df = df[df['Y.pred'] == 'DNAmB2M']\n", "df['feature'] = df['var']\n", "df['coefficient'] = df['beta']\n", "model.features = ['age'] + df['feature'][2:].tolist()" ] }, { "cell_type": "code", "execution_count": 24, "id": "07d919cb", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Y.predvarbetafeaturecoefficient
140DNAmB2MIntercept1.412953e+06Intercept1.412953e+06
141DNAmB2MAge1.048692e+04Age1.048692e+04
142DNAmB2Mcg139473173.169626e+05cg139473173.169626e+05
143DNAmB2Mcg225101393.392710e+04cg225101393.392710e+04
144DNAmB2Mcg10116490-1.606124e+05cg10116490-1.606124e+05
\n", "
" ], "text/plain": [ " Y.pred var beta feature coefficient\n", "140 DNAmB2M Intercept 1.412953e+06 Intercept 1.412953e+06\n", "141 DNAmB2M Age 1.048692e+04 Age 1.048692e+04\n", "142 DNAmB2M cg13947317 3.169626e+05 cg13947317 3.169626e+05\n", "143 DNAmB2M cg22510139 3.392710e+04 cg22510139 3.392710e+04\n", "144 DNAmB2M cg10116490 -1.606124e+05 cg10116490 -1.606124e+05" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.head()" ] }, { "cell_type": "markdown", "id": "ee6d8fa0-4767-4c45-9717-eb1c95e2ddc0", "metadata": {}, "source": [ "## Load weights into base model" ] }, { "cell_type": "markdown", "id": "3958ba73-42e8-40a5-94a1-4f4b8ae05dca", "metadata": {}, "source": [ "#### Linear model" ] }, { "cell_type": "code", "execution_count": 25, "id": "321a437c-8888-4e10-96e9-5ed2826a8f74", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.647294Z", "iopub.status.busy": "2025-04-07T17:51:44.647116Z", "iopub.status.idle": "2025-04-07T17:51:44.688112Z", "shell.execute_reply": "2025-04-07T17:51:44.687757Z" } }, "outputs": [], "source": [ "weights = torch.tensor(df['coefficient'][1:].tolist()).unsqueeze(0)\n", "intercept = torch.tensor([df['coefficient'].iloc[0]])" ] }, { "cell_type": "markdown", "id": "5742dc16-e063-414f-a38e-9721beb11351", "metadata": {}, "source": [ "#### Linear model" ] }, { "cell_type": "code", "execution_count": 26, "id": "c2e54115-c17b-48ce-88f1-de546c90d2b3", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.689783Z", "iopub.status.busy": "2025-04-07T17:51:44.689683Z", "iopub.status.idle": "2025-04-07T17:51:44.692010Z", "shell.execute_reply": "2025-04-07T17:51:44.691738Z" } }, "outputs": [], "source": [ "base_model = pya.models.LinearModel(input_dim=len(model.features))\n", "\n", "base_model.linear.weight.data = weights.float()\n", "base_model.linear.bias.data = intercept.float()\n", "\n", "model.base_model = base_model" ] }, { "cell_type": "markdown", "id": "ad8b4c1d-9d57-48b7-9a30-bcfea7b747b1", "metadata": {}, "source": [ "## Load reference values" ] }, { "cell_type": "code", "execution_count": 27, "id": "2089b66f-9cc4-4528-9bdc-5e45efc6d06b", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.693409Z", "iopub.status.busy": "2025-04-07T17:51:44.693309Z", "iopub.status.idle": "2025-04-07T17:51:44.708180Z", "shell.execute_reply": "2025-04-07T17:51:44.707878Z" } }, "outputs": [], "source": [ "reference_df = pd.read_csv('datMiniAnnotation3_Gold.csv', index_col=0)\n", "model.reference_values = [65] + reference_df.loc[model.features[1:]]['gold'].tolist()" ] }, { "cell_type": "markdown", "id": "af3bcf7b-74a8-4d21-9ccb-4de0c2b0516b", "metadata": {}, "source": [ "## Load preprocess and postprocess objects" ] }, { "cell_type": "code", "execution_count": 28, "id": "7a22fb20-c605-424d-8efb-7620c2c0755c", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.709686Z", "iopub.status.busy": "2025-04-07T17:51:44.709594Z", "iopub.status.idle": "2025-04-07T17:51:44.711225Z", "shell.execute_reply": "2025-04-07T17:51:44.710973Z" } }, "outputs": [], "source": [ "model.preprocess_name = None\n", "model.preprocess_dependencies = None" ] }, { "cell_type": "code", "execution_count": 29, "id": "ff4a21cb-cf41-44dc-9ed1-95cf8aa15772", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.712430Z", "iopub.status.busy": "2025-04-07T17:51:44.712349Z", "iopub.status.idle": "2025-04-07T17:51:44.713842Z", "shell.execute_reply": "2025-04-07T17:51:44.713585Z" } }, "outputs": [], "source": [ "model.postprocess_name = None\n", "model.postprocess_dependencies = None" ] }, { "cell_type": "markdown", "id": "86e3d6b1-e67e-4f3d-bd39-0ebec5726c3c", "metadata": {}, "source": [ "## Check all clock parameters" ] }, { "cell_type": "code", "execution_count": 30, "id": "2168355c-47d9-475d-b816-49f65e74887c", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.715112Z", "iopub.status.busy": "2025-04-07T17:51:44.715032Z", "iopub.status.idle": "2025-04-07T17:51:44.726874Z", "shell.execute_reply": "2025-04-07T17:51:44.726577Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "%==================================== Model Details ====================================%\n", "Model Attributes:\n", "\n", "training: True\n", "metadata: {'approved_by_author': '⌛',\n", " 'citation': 'Lu, Ake T., et al. \"DNA methylation GrimAge version 2.\" Aging '\n", " '(Albany NY) 14.23 (2022): 9484.',\n", " 'clock_name': 'grimage2b2m',\n", " 'data_type': 'methylation',\n", " 'doi': 'https://doi.org/10.18632/aging.204434',\n", " 'notes': None,\n", " 'research_only': True,\n", " 'species': 'Homo sapiens',\n", " 'version': None,\n", " 'year': 2022}\n", "reference_values: [65, 0.945983138574711, 0.481093763685811, 0.27748709868544, 0.951672124097172, 0.468094302501278, 0.563678993728847, 0.948817639234683, 0.90429743695085, 0.0370947881546007, 0.0264846427378418, 0.933625606274069, 0.806897977714167, 0.616012873042032, 0.92495320469147, 0.905370920309603, 0.927267360866437, 0.949349810160961, 0.747531904904077, 0.949456556741532, 0.53813155435689, 0.71825377235117, 0.653576107061525, 0.747645234935016, 0.818024480938858, 0.622775695948335, 0.949645872518435, 0.572473769572986, 0.0359141899481796, 0.953339533069151]... [Total elements: 92]\n", "preprocess_name: None\n", "preprocess_dependencies: None\n", "postprocess_name: 'cox_to_years'\n", "postprocess_dependencies: None\n", "features: ['age', 'cg13947317', 'cg22510139', 'cg10116490', 'cg27396830', 'cg19414383', 'cg03345668', 'cg17719473', 'cg20569940', 'cg26156167', 'cg10578779', 'cg15631106', 'cg22704788', 'cg20240347', 'cg17803430', 'cg20800892', 'cg01909777', 'cg15849439', 'cg12605080', 'cg01163330', 'cg10753966', 'cg19501902', 'cg13287247', 'cg22930808', 'cg08122652', 'cg00959259', 'cg20576510', 'cg10202557', 'cg27134386', 'cg10584300']... [Total elements: 92]\n", "base_model_features: None\n", "\n", "%==================================== Model Details ====================================%\n", "Model Structure:\n", "\n", "base_model: LinearModel(\n", " (linear): Linear(in_features=92, out_features=1, bias=True)\n", ")\n", "\n", "%==================================== Model Details ====================================%\n", "Model Parameters and Weights:\n", "\n", "base_model.linear.weight: [10486.9150390625, 316962.65625, 33927.10546875, -160612.375, 75457.171875, 87985.7421875, 292882.90625, -23280.169921875, 43791.1796875, 302011.96875, 1916187.0, -58500.59375, -126869.8828125, 1506.065185546875, 1417.4544677734375, 44895.46875, 267379.5625, -924930.5625, 69711.0390625, 102607.9921875, 49483.265625, 65359.765625, -13569.71875, -13531.2998046875, -84787.703125, -129131.7265625, 412413.875, -67296.7265625, 28426.35546875, 89744.1875]... [Tensor of shape torch.Size([1, 92])]\n", "base_model.linear.bias: tensor([1412953.3750])\n", "\n", "%==================================== Model Details ====================================%\n", "\n" ] } ], "source": [ "pya.utils.print_model_details(model)" ] }, { "cell_type": "markdown", "id": "986d0262-e0c7-4036-b687-dee53ba392fb", "metadata": {}, "source": [ "## Basic test" ] }, { "cell_type": "code", "execution_count": 31, "id": "936b9877-d076-4ced-99aa-e8d4c58c5caf", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.728239Z", "iopub.status.busy": "2025-04-07T17:51:44.728153Z", "iopub.status.idle": "2025-04-07T17:51:44.733560Z", "shell.execute_reply": "2025-04-07T17:51:44.733262Z" } }, "outputs": [ { "data": { "text/plain": [ "tensor([[ 4694786.7644],\n", " [-1597495.9568],\n", " [ 3826313.5179],\n", " [ 5135406.7161],\n", " [-1016183.9231],\n", " [ 4471339.9569],\n", " [-1899905.5050],\n", " [ 7716734.0988],\n", " [ 5404589.6838],\n", " [ 2730438.6117]], dtype=torch.float64, grad_fn=)" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch.manual_seed(42)\n", "input = torch.randn(10, len(model.features), dtype=float).double()\n", "model.eval()\n", "model.to(float)\n", "pred = model(input)\n", "pred" ] }, { "cell_type": "markdown", "id": "fe8299d7-9285-4e22-82fd-b664434b4369", "metadata": {}, "source": [ "## Save torch model" ] }, { "cell_type": "code", "execution_count": 32, "id": "5ef2fa8d-c80b-4fdd-8555-79c0d541788e", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.734916Z", "iopub.status.busy": "2025-04-07T17:51:44.734814Z", "iopub.status.idle": "2025-04-07T17:51:44.739278Z", "shell.execute_reply": "2025-04-07T17:51:44.738936Z" } }, "outputs": [], "source": [ "torch.save(model, f\"../weights/{model.metadata['clock_name']}.pt\")" ] }, { "cell_type": "markdown", "id": "bac6257b-8d08-4a90-8d0b-7f745dc11ac1", "metadata": {}, "source": [ "## Clear directory\n", "" ] }, { "cell_type": "code", "execution_count": 33, "id": "11aeaa70-44c0-42f9-86d7-740e3849a7a6", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.740582Z", "iopub.status.busy": "2025-04-07T17:51:44.740494Z", "iopub.status.idle": "2025-04-07T17:51:44.743819Z", "shell.execute_reply": "2025-04-07T17:51:44.743572Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Deleted file: grimage2_subcomponents.csv\n", "Deleted file: datMiniAnnotation3_Gold.csv\n", "Deleted file: grimage2.csv\n" ] } ], "source": [ "# Function to remove a folder and all its contents\n", "def remove_folder(path):\n", " try:\n", " shutil.rmtree(path)\n", " print(f\"Deleted folder: {path}\")\n", " except Exception as e:\n", " print(f\"Error deleting folder {path}: {e}\")\n", "\n", "# Get a list of all files and folders in the current directory\n", "all_items = os.listdir('.')\n", "\n", "# Loop through the items\n", "for item in all_items:\n", " # Check if it's a file and does not end with .ipynb\n", " if os.path.isfile(item) and not item.endswith('.ipynb'):\n", " os.remove(item)\n", " print(f\"Deleted file: {item}\")\n", " # Check if it's a folder\n", " elif os.path.isdir(item):\n", " remove_folder(item)" ] } ], "metadata": { "kernelspec": { "display_name": "research", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.7" } }, "nbformat": 4, "nbformat_minor": 5 }