{ "cells": [ { "cell_type": "markdown", "id": "2f04eee0-5928-4e74-a754-6dc2e528810c", "metadata": {}, "source": [ "# GrimAge2GDF15" ] }, { "cell_type": "markdown", "id": "a3f514a3-772c-4a14-afdf-5a8376851ff4", "metadata": {}, "source": [ "## Index\n", "1. [Instantiate model class](#Instantiate-model-class)\n", "2. [Define clock metadata](#Define-clock-metadata)\n", "3. [Download clock dependencies](#Download-clock-dependencies)\n", "5. [Load features](#Load-features)\n", "6. [Load weights into base model](#Load-weights-into-base-model)\n", "7. [Load reference values](#Load-reference-values)\n", "8. [Load preprocess and postprocess objects](#Load-preprocess-and-postprocess-objects)\n", "10. [Check all clock parameters](#Check-all-clock-parameters)\n", "10. [Basic test](#Basic-test)\n", "11. [Save torch model](#Save-torch-model)\n", "12. [Clear directory](#Clear-directory)" ] }, { "cell_type": "markdown", "id": "d95fafdc-643a-40ea-a689-200bd132e90c", "metadata": {}, "source": [ "Let's first import some packages:" ] }, { "cell_type": "code", "execution_count": 17, "id": "4adfb4de-cd79-4913-a1af-9e23e9e236c9", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:41.316681Z", "iopub.status.busy": "2025-04-07T17:51:41.316440Z", "iopub.status.idle": "2025-04-07T17:51:42.738147Z", "shell.execute_reply": "2025-04-07T17:51:42.737780Z" } }, "outputs": [], "source": [ "import os\n", "import inspect\n", "import shutil\n", "import json\n", "import torch\n", "import pandas as pd\n", "import pyaging as pya\n", "import numpy as np" ] }, { "cell_type": "markdown", "id": "145082e5-ced4-47ae-88c0-cb69773e3c5a", "metadata": {}, "source": [ "## Instantiate model class" ] }, { "cell_type": "code", "execution_count": 18, "id": "8aa77372-7ed3-4da7-abc9-d30372106139", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:42.740018Z", "iopub.status.busy": "2025-04-07T17:51:42.739761Z", "iopub.status.idle": "2025-04-07T17:51:42.750935Z", "shell.execute_reply": "2025-04-07T17:51:42.750574Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "class GrimAge2GDF15(pyagingModel):\n", " def __init__(self):\n", " super().__init__()\n", "\n", " def preprocess(self, x):\n", " return x\n", "\n", " def postprocess(self, x):\n", " return x\n", "\n" ] } ], "source": [ "def print_entire_class(cls):\n", " source = inspect.getsource(cls)\n", " print(source)\n", "\n", "print_entire_class(pya.models.GrimAge2GDF15)" ] }, { "cell_type": "code", "execution_count": 19, "id": "78536494-f1d9-44de-8583-c89a310d2307", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:42.753015Z", "iopub.status.busy": "2025-04-07T17:51:42.752877Z", "iopub.status.idle": "2025-04-07T17:51:42.754599Z", "shell.execute_reply": "2025-04-07T17:51:42.754314Z" } }, "outputs": [], "source": [ "model = pya.models.GrimAge2GDF15()" ] }, { "cell_type": "markdown", "id": "51f8615e-01fa-4aa5-b196-3ee2b35d261c", "metadata": {}, "source": [ "## Define clock metadata" ] }, { "cell_type": "code", "execution_count": 20, "id": "135ce001-03f7-4025-bceb-01a3e2e2b0ef", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:42.755964Z", "iopub.status.busy": "2025-04-07T17:51:42.755873Z", "iopub.status.idle": "2025-04-07T17:51:42.758035Z", "shell.execute_reply": "2025-04-07T17:51:42.757775Z" } }, "outputs": [], "source": [ "model.metadata[\"clock_name\"] = \"grimage2gdf15\"\n", "model.metadata[\"data_type\"] = \"DNA methylation\" # Paper: Blood DNA methylation profiles were measured to construct the biomarkers.\n", "model.metadata[\"species\"] = \"Homo sapiens\" # Paper: The training set comprised human FHS Offspring participants.\n", "model.metadata[\"year\"] = 2022\n", "model.metadata[\"approved_by_author\"] = \"⌛\"\n", "model.metadata[\"citation\"] = \"Lu, A. T., et al. “DNA methylation GrimAge version 2.” Aging 14(23): 9484–9549 (2022).\"\n", "model.metadata[\"doi\"] = \"https://doi.org/10.18632/aging.204434\"\n", "model.metadata[\"notes\"] = \"GDF-15 is implicated in aging and mitochondrial dysfunction; DNAm GDF-15 is a GrimAge component.\"\n", "model.metadata[\"research_only\"] = True\n", "model.metadata[\"tissue\"] = [\"whole blood\"] # Paper: Peripheral blood/buffy-coat DNA from FHS exam 8 supplied the methylation predictors.\n", "model.metadata[\"predicts\"] = [\"GDF-15\"] # Paper: DNAm growth differentiation factor 15 is a DNAm-based surrogate of the corresponding observed biomarker.\n", "model.metadata[\"training_target\"] = [\"GDF-15\"] # Paper: Growth Differentiation Factor 15 was one of the observed plasma-protein or smoking variables estimated by a DNAm surrogate.\n", "model.metadata[\"unit\"] = [\"picograms per milliliter\"] # Paper: The inherited GrimAge plasma-protein targets were immunoassay measurements in pg/mL.\n", "model.metadata[\"model_type\"] = \"elastic net regression\" # Paper: Stage-1 DNAm surrogates were fit by elastic-net regression with ten-fold cross-validation.\n", "model.metadata[\"platform\"] = [\"Illumina 450K\"] # Paper: Training methylation profiling was based on the Illumina HumanMethylation450K BeadChip.\n", "model.metadata[\"population\"] = \"older adults\" # Paper: GrimAge2 was trained in 1,833 FHS participants aged 40–92 years.\n", "model.metadata[\"journal\"] = \"Aging\"\n", "model.metadata[\"last_author\"] = \"Steve Horvath\"\n", "model.metadata[\"n_features\"] = 138\n", "model.metadata[\"citations\"] = 291\n", "model.metadata[\"citations_date\"] = \"2026-07-05\"\n" ] }, { "cell_type": "markdown", "id": "74492239-5aae-4026-9d90-6bc9c574c110", "metadata": {}, "source": [ "## Download clock dependencies" ] }, { "cell_type": "code", "execution_count": 21, "id": "95f6ba57", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "|-----------> Downloading data to ./grimage2_subcomponents.csv\n", "|-----------> in progress: 100.0000%\n", "|-----------> Downloading data to ./grimage2.csv\n", "|-----------> in progress: 100.0000%%\n", "|-----------> Downloading data to ./datMiniAnnotation3_Gold.csv\n", "|-----------> in progress: 100.0000%\n" ] } ], "source": [ "logger = pya.logger.Logger()\n", "urls = [\n", " \"https://huggingface.co/lucascamillomd/pyaging-data/resolve/main/supporting_files/grimage2_subcomponents.csv\",\n", " \"https://huggingface.co/lucascamillomd/pyaging-data/resolve/main/supporting_files/grimage2.csv\",\n", " \"https://huggingface.co/lucascamillomd/pyaging-data/resolve/main/supporting_files/datMiniAnnotation3_Gold.csv\",\n", "]\n", "dir = \".\"\n", "for url in urls:\n", " pya.utils.download(url, dir, logger, indent_level=1)" ] }, { "cell_type": "markdown", "id": "a14c7fc1-abe5-42a3-8bc9-0987521ddf33", "metadata": {}, "source": [ "## Load features" ] }, { "cell_type": "markdown", "id": "3e737582-3a28-4f55-8da9-3e34125362cc", "metadata": {}, "source": [ "#### From CSV" ] }, { "cell_type": "code", "execution_count": 22, "id": "f1486db4", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array(['DNAmGDF_15', 'DNAmB2M', 'DNAmCystatin_C', 'DNAmTIMP_1', 'DNAmadm',\n", " 'DNAmpai_1', 'DNAmleptin', 'DNAmPACKYRS', 'DNAmlog.CRP',\n", " 'DNAmlog.A1C'], dtype=object)" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df = pd.read_csv('grimage2_subcomponents.csv', index_col=0)\n", "df['Y.pred'].unique()" ] }, { "cell_type": "code", "execution_count": 23, "id": "f7ad18f4", "metadata": {}, "outputs": [], "source": [ "df = df[df['Y.pred'] == 'DNAmGDF_15']\n", "df['feature'] = df['var']\n", "df['coefficient'] = df['beta']\n", "model.features = ['age'] + df['feature'][2:].tolist()" ] }, { "cell_type": "code", "execution_count": 24, "id": "880e30cb", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Y.predvarbetafeaturecoefficient
1DNAmGDF_15Intercept1975.798380Intercept1975.798380
2DNAmGDF_15Age9.351357Age9.351357
3DNAmGDF_15cg2178994184.364575cg2178994184.364575
4DNAmGDF_15cg22866430143.696062cg22866430143.696062
5DNAmGDF_15cg1823497381.378646cg1823497381.378646
\n", "
" ], "text/plain": [ " Y.pred var beta feature coefficient\n", "1 DNAmGDF_15 Intercept 1975.798380 Intercept 1975.798380\n", "2 DNAmGDF_15 Age 9.351357 Age 9.351357\n", "3 DNAmGDF_15 cg21789941 84.364575 cg21789941 84.364575\n", "4 DNAmGDF_15 cg22866430 143.696062 cg22866430 143.696062\n", "5 DNAmGDF_15 cg18234973 81.378646 cg18234973 81.378646" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.head()" ] }, { "cell_type": "markdown", "id": "ee6d8fa0-4767-4c45-9717-eb1c95e2ddc0", "metadata": {}, "source": [ "## Load weights into base model" ] }, { "cell_type": "markdown", "id": "3958ba73-42e8-40a5-94a1-4f4b8ae05dca", "metadata": {}, "source": [ "#### Linear model" ] }, { "cell_type": "code", "execution_count": 25, "id": "321a437c-8888-4e10-96e9-5ed2826a8f74", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.647294Z", "iopub.status.busy": "2025-04-07T17:51:44.647116Z", "iopub.status.idle": "2025-04-07T17:51:44.688112Z", "shell.execute_reply": "2025-04-07T17:51:44.687757Z" } }, "outputs": [], "source": [ "weights = torch.tensor(df['coefficient'][1:].tolist()).unsqueeze(0)\n", "intercept = torch.tensor([df['coefficient'].iloc[0]])" ] }, { "cell_type": "markdown", "id": "5742dc16-e063-414f-a38e-9721beb11351", "metadata": {}, "source": [ "#### Linear model" ] }, { "cell_type": "code", "execution_count": 26, "id": "c2e54115-c17b-48ce-88f1-de546c90d2b3", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.689783Z", "iopub.status.busy": "2025-04-07T17:51:44.689683Z", "iopub.status.idle": "2025-04-07T17:51:44.692010Z", "shell.execute_reply": "2025-04-07T17:51:44.691738Z" } }, "outputs": [], "source": [ "base_model = pya.models.LinearModel(input_dim=len(model.features))\n", "\n", "base_model.linear.weight.data = weights.float()\n", "base_model.linear.bias.data = intercept.float()\n", "\n", "model.base_model = base_model" ] }, { "cell_type": "markdown", "id": "ad8b4c1d-9d57-48b7-9a30-bcfea7b747b1", "metadata": {}, "source": [ "## Load reference values" ] }, { "cell_type": "code", "execution_count": 27, "id": "2089b66f-9cc4-4528-9bdc-5e45efc6d06b", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.693409Z", "iopub.status.busy": "2025-04-07T17:51:44.693309Z", "iopub.status.idle": "2025-04-07T17:51:44.708180Z", "shell.execute_reply": "2025-04-07T17:51:44.707878Z" } }, "outputs": [], "source": [ "reference_df = pd.read_csv('datMiniAnnotation3_Gold.csv', index_col=0)\n", "model.reference_values = [65] + reference_df.loc[model.features[1:]]['gold'].tolist()" ] }, { "cell_type": "markdown", "id": "af3bcf7b-74a8-4d21-9ccb-4de0c2b0516b", "metadata": {}, "source": [ "## Load preprocess and postprocess objects" ] }, { "cell_type": "code", "execution_count": 28, "id": "7a22fb20-c605-424d-8efb-7620c2c0755c", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.709686Z", "iopub.status.busy": "2025-04-07T17:51:44.709594Z", "iopub.status.idle": "2025-04-07T17:51:44.711225Z", "shell.execute_reply": "2025-04-07T17:51:44.710973Z" } }, "outputs": [], "source": [ "model.preprocess_name = None\n", "model.preprocess_dependencies = None" ] }, { "cell_type": "code", "execution_count": 29, "id": "ff4a21cb-cf41-44dc-9ed1-95cf8aa15772", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.712430Z", "iopub.status.busy": "2025-04-07T17:51:44.712349Z", "iopub.status.idle": "2025-04-07T17:51:44.713842Z", "shell.execute_reply": "2025-04-07T17:51:44.713585Z" } }, "outputs": [], "source": [ "model.postprocess_name = None\n", "model.postprocess_dependencies = None" ] }, { "cell_type": "markdown", "id": "86e3d6b1-e67e-4f3d-bd39-0ebec5726c3c", "metadata": {}, "source": [ "## Check all clock parameters" ] }, { "cell_type": "code", "execution_count": 30, "id": "2168355c-47d9-475d-b816-49f65e74887c", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.715112Z", "iopub.status.busy": "2025-04-07T17:51:44.715032Z", "iopub.status.idle": "2025-04-07T17:51:44.726874Z", "shell.execute_reply": "2025-04-07T17:51:44.726577Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "%==================================== Model Details ====================================%\n", "Model Attributes:\n", "\n", "training: True\n", "metadata: {'approved_by_author': '⌛',\n", " 'citation': 'Lu, Ake T., et al. \"DNA methylation GrimAge version 2.\" Aging '\n", " '(Albany NY) 14.23 (2022): 9484.',\n", " 'clock_name': 'grimage2gdf15',\n", " 'data_type': 'methylation',\n", " 'doi': 'https://doi.org/10.18632/aging.204434',\n", " 'notes': None,\n", " 'research_only': True,\n", " 'species': 'Homo sapiens',\n", " 'version': None,\n", " 'year': 2022}\n", "reference_values: [65, 0.401307658505067, 0.860293923434293, 0.947280006512448, 0.930527165222189, 0.255565869348342, 0.0264846427378418, 0.902879804270564, 0.941001569709992, 0.694086158029063, 0.915468273729728, 0.592724594624117, 0.0447197866177092, 0.938589245529331, 0.441107352002649, 0.584475440293952, 0.908361953693711, 0.830024180811995, 0.687215910925347, 0.919973258903412, 0.949349810160961, 0.931787690661021, 0.0689538820445645, 0.703634507571187, 0.912810466343666, 0.946549875970756, 0.949645872518435, 0.0614416243373215, 0.934361053755199, 0.815829866270951]... [Total elements: 138]\n", "preprocess_name: None\n", "preprocess_dependencies: None\n", "postprocess_name: 'cox_to_years'\n", "postprocess_dependencies: None\n", "features: ['age', 'cg21789941', 'cg22866430', 'cg18234973', 'cg26110733', 'cg10636246', 'cg10578779', 'cg17591574', 'cg13371627', 'cg12458003', 'cg13180152', 'cg21459583', 'cg18278247', 'cg06022832', 'cg21566642', 'cg01940273', 'cg08110610', 'cg00684178', 'cg27380813', 'cg03827386', 'cg15849439', 'cg06349851', 'cg25598319', 'cg25845889', 'cg15228492', 'cg03900798', 'cg20576510', 'cg12988996', 'cg11569478', 'cg01603456']... [Total elements: 138]\n", "base_model_features: None\n", "\n", "%==================================== Model Details ====================================%\n", "Model Structure:\n", "\n", "base_model: LinearModel(\n", " (linear): Linear(in_features=138, out_features=1, bias=True)\n", ")\n", "\n", "%==================================== Model Details ====================================%\n", "Model Parameters and Weights:\n", "\n", "base_model.linear.weight: [9.351357460021973, 84.36457824707031, 143.69606018066406, 81.37864685058594, 29.040103912353516, -112.49447631835938, 11.24372673034668, -118.38355255126953, -1980.7607421875, 12.090482711791992, 381.68292236328125, 20.09428596496582, 79.332275390625, 93.16657257080078, -42.04895782470703, -56.21128463745117, 50.33696746826172, -4.690526962280273, 23.865774154663086, 110.17974090576172, -504.0067138671875, 454.8924255371094, 3.939922571182251, 10.757635116577148, -21.75938606262207, 191.6622314453125, 342.773193359375, -183.74392700195312, 297.9110107421875, 50.03202438354492]... [Tensor of shape torch.Size([1, 138])]\n", "base_model.linear.bias: tensor([1975.7983])\n", "\n", "%==================================== Model Details ====================================%\n", "\n" ] } ], "source": [ "pya.utils.print_model_details(model)" ] }, { "cell_type": "markdown", "id": "986d0262-e0c7-4036-b687-dee53ba392fb", "metadata": {}, "source": [ "## Basic test" ] }, { "cell_type": "code", "execution_count": 31, "id": "936b9877-d076-4ced-99aa-e8d4c58c5caf", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.728239Z", "iopub.status.busy": "2025-04-07T17:51:44.728153Z", "iopub.status.idle": "2025-04-07T17:51:44.733560Z", "shell.execute_reply": "2025-04-07T17:51:44.733262Z" } }, "outputs": [ { "data": { "text/plain": [ "tensor([[ 1552.1130],\n", " [ 4822.7574],\n", " [-4143.4484],\n", " [-1189.7159],\n", " [-3966.4775],\n", " [ 3561.1223],\n", " [-3886.9276],\n", " [ 9188.9426],\n", " [ 7062.6772],\n", " [21668.6671]], dtype=torch.float64, grad_fn=)" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch.manual_seed(42)\n", "input = torch.randn(10, len(model.features), dtype=float).double()\n", "model.eval()\n", "model.to(float)\n", "pred = model(input)\n", "pred" ] }, { "cell_type": "markdown", "id": "fe8299d7-9285-4e22-82fd-b664434b4369", "metadata": {}, "source": [ "## Save torch model" ] }, { "cell_type": "code", "execution_count": 32, "id": "5ef2fa8d-c80b-4fdd-8555-79c0d541788e", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.734916Z", "iopub.status.busy": "2025-04-07T17:51:44.734814Z", "iopub.status.idle": "2025-04-07T17:51:44.739278Z", "shell.execute_reply": "2025-04-07T17:51:44.738936Z" } }, "outputs": [], "source": [ "torch.save(model, f\"../weights/{model.metadata['clock_name']}.pt\")" ] }, { "cell_type": "markdown", "id": "bac6257b-8d08-4a90-8d0b-7f745dc11ac1", "metadata": {}, "source": [ "## Clear directory\n", "" ] }, { "cell_type": "code", "execution_count": 33, "id": "11aeaa70-44c0-42f9-86d7-740e3849a7a6", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:44.740582Z", "iopub.status.busy": "2025-04-07T17:51:44.740494Z", "iopub.status.idle": "2025-04-07T17:51:44.743819Z", "shell.execute_reply": "2025-04-07T17:51:44.743572Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Deleted file: grimage2_subcomponents.csv\n", "Deleted file: datMiniAnnotation3_Gold.csv\n", "Deleted file: grimage2.csv\n" ] } ], "source": [ "# Function to remove a folder and all its contents\n", "def remove_folder(path):\n", " try:\n", " shutil.rmtree(path)\n", " print(f\"Deleted folder: {path}\")\n", " except Exception as e:\n", " print(f\"Error deleting folder {path}: {e}\")\n", "\n", "# Get a list of all files and folders in the current directory\n", "all_items = os.listdir('.')\n", "\n", "# Loop through the items\n", "for item in all_items:\n", " # Check if it's a file and does not end with .ipynb\n", " if os.path.isfile(item) and not item.endswith('.ipynb'):\n", " os.remove(item)\n", " print(f\"Deleted file: {item}\")\n", " # Check if it's a folder\n", " elif os.path.isdir(item):\n", " remove_folder(item)" ] } ], "metadata": { "kernelspec": { "display_name": "research", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.7" } }, "nbformat": 4, "nbformat_minor": 5 }