{ "cells": [ { "cell_type": "markdown", "id": "2f04eee0-5928-4e74-a754-6dc2e528810c", "metadata": {}, "source": [ "# MammalianBlood2" ] }, { "cell_type": "markdown", "id": "a3f514a3-772c-4a14-afdf-5a8376851ff4", "metadata": {}, "source": [ "## Index\n", "1. [Instantiate model class](#Instantiate-model-class)\n", "2. [Define clock metadata](#Define-clock-metadata)\n", "3. [Download clock dependencies](#Download-clock-dependencies)\n", "5. [Load features](#Load-features)\n", "6. [Load weights into base model](#Load-weights-into-base-model)\n", "7. [Load reference values](#Load-reference-values)\n", "8. [Load preprocess and postprocess objects](#Load-preprocess-and-postprocess-objects)\n", "10. [Check all clock parameters](#Check-all-clock-parameters)\n", "10. [Basic test](#Basic-test)\n", "11. [Save torch model](#Save-torch-model)\n", "12. [Clear directory](#Clear-directory)" ] }, { "cell_type": "markdown", "id": "d95fafdc-643a-40ea-a689-200bd132e90c", "metadata": {}, "source": [ "Let's first import some packages:" ] }, { "cell_type": "code", "execution_count": 1, "id": "4adfb4de-cd79-4913-a1af-9e23e9e236c9", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:58:10.196194Z", "iopub.status.busy": "2025-04-07T17:58:10.195788Z", "iopub.status.idle": "2025-04-07T17:58:11.671668Z", "shell.execute_reply": "2025-04-07T17:58:11.671349Z" } }, "outputs": [], "source": [ "import os\n", "import inspect\n", "import shutil\n", "import json\n", "import torch\n", "import pandas as pd\n", "import pyaging as pya\n", "import numpy as np" ] }, { "cell_type": "markdown", "id": "145082e5-ced4-47ae-88c0-cb69773e3c5a", "metadata": {}, "source": [ "## Instantiate model class" ] }, { "cell_type": "code", "execution_count": 2, "id": "8aa77372-7ed3-4da7-abc9-d30372106139", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:58:11.673404Z", "iopub.status.busy": "2025-04-07T17:58:11.673170Z", "iopub.status.idle": "2025-04-07T17:58:11.681231Z", "shell.execute_reply": "2025-04-07T17:58:11.680976Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "class MammalianBlood2(pyagingModel):\n", " def __init__(self):\n", " super().__init__()\n", "\n", " def forward(self, x):\n", " x_cpg = x[:, :-1756] # number of species in lookup table\n", " x_species = x[:, -1756:] # number of species in lookup table\n", " x = self.base_model(x_cpg)\n", " x = self.postprocess(x, x_species)\n", " return x\n", "\n", " def preprocess(self, x):\n", " return x\n", "\n", " def postprocess(self, x, x_species):\n", " \"\"\"\n", " Converts output of relative age to age in units of years.\n", " \"\"\"\n", " indices = torch.argmax(x_species, dim=1)\n", " anage_array = self.postprocess_dependencies[0]\n", " anage_tensor = torch.tensor(anage_array, dtype=x.dtype, device=x.device)\n", " gestation_time = anage_tensor[indices, 0].unsqueeze(1)\n", " max_age = anage_tensor[indices, 3].unsqueeze(1)\n", "\n", " x = torch.exp(-torch.exp(-x))\n", " x = x * (max_age + gestation_time) - gestation_time\n", " return x\n", "\n" ] } ], "source": [ "def print_entire_class(cls):\n", " source = inspect.getsource(cls)\n", " print(source)\n", "\n", "print_entire_class(pya.models.MammalianBlood2)" ] }, { "cell_type": "code", "execution_count": 3, "id": "78536494-f1d9-44de-8583-c89a310d2307", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:58:11.682439Z", "iopub.status.busy": "2025-04-07T17:58:11.682349Z", "iopub.status.idle": "2025-04-07T17:58:11.683957Z", "shell.execute_reply": "2025-04-07T17:58:11.683718Z" } }, "outputs": [], "source": [ "model = pya.models.MammalianBlood2()" ] }, { "cell_type": "markdown", "id": "51f8615e-01fa-4aa5-b196-3ee2b35d261c", "metadata": {}, "source": [ "## Define clock metadata" ] }, { "cell_type": "code", "execution_count": 4, "id": "6601da9e-8adc-44ee-9308-75e3cd31b816", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:58:11.685197Z", "iopub.status.busy": "2025-04-07T17:58:11.685105Z", "iopub.status.idle": "2025-04-07T17:58:11.687127Z", "shell.execute_reply": "2025-04-07T17:58:11.686874Z" } }, "outputs": [], "source": [ "model.metadata[\"clock_name\"] = \"mammalianblood2\"\n", "model.metadata[\"data_type\"] = \"DNA methylation\" # Paper: methylation\n", "model.metadata[\"species\"] = \"multiple species\" # Paper: multi\n", "model.metadata[\"year\"] = 2023\n", "model.metadata[\"approved_by_author\"] = \"⌛\"\n", "model.metadata[\"citation\"] = \"Lu, A. T., et al. \\\"Universal DNA methylation age across mammalian tissues.\\\" Nature Aging 3.9 (2023): 1144-1166.\"\n", "model.metadata[\"doi\"] = \"https://doi.org/10.1038/s43587-023-00462-6\"\n", "model.metadata[\"notes\"] = \"Blood-specific universal clock 2 fits relative age and returns species-adjusted chronological age in years after the maximum-lifespan inverse transformation.\"\n", "model.metadata[\"research_only\"] = None\n", "model.metadata[\"tissue\"] = [\"blood\"] # Paper: blood\n", "model.metadata[\"predicts\"] = [\"chronological age\"] # Paper: The official inverse transformation returns DNAm chronological age in years.\n", "model.metadata[\"training_target\"] = [\"relative age\"] # Paper: relative age\n", "model.metadata[\"unit\"] = [\"years\"] # Paper: The official inverse transformation returns DNAm chronological age in years.\n", "model.metadata[\"model_type\"] = \"elastic net regression\" # Paper: Elastic net\n", "model.metadata[\"platform\"] = [\"Horvath MammalMethylChip40\"] # Paper: HorvathMammalMethylChip40\n", "model.metadata[\"population\"] = \"multiple mammalian species\" # Paper: multi-species mammalian blood samples\n", "model.metadata[\"journal\"] = \"Nature Aging\"\n", "model.metadata[\"last_author\"] = \"Steve Horvath\"\n", "model.metadata[\"n_features\"] = 2257\n", "model.metadata[\"citations\"] = 390\n", "model.metadata[\"citations_date\"] = \"2026-07-05\"\n" ] }, { "cell_type": "markdown", "id": "74492239-5aae-4026-9d90-6bc9c574c110", "metadata": {}, "source": [ "## Download clock dependencies" ] }, { "cell_type": "markdown", "id": "5881f33e-0486-45d1-bd87-9c995f47ca62", "metadata": {}, "source": [ "#### Download GitHub repository" ] }, { "cell_type": "code", "execution_count": 5, "id": "e85525f7-6da4-4962-a7d5-0607e76eea33", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:58:11.688461Z", "iopub.status.busy": "2025-04-07T17:58:11.688384Z", "iopub.status.idle": "2025-04-07T18:00:19.988633Z", "shell.execute_reply": "2025-04-07T18:00:19.988299Z" } }, "outputs": [ { "data": { "text/plain": [ "0" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "github_url = \"https://github.com/shorvath/MammalianMethylationConsortium.git\"\n", "github_folder_name = github_url.split('/')[-1].split('.')[0]\n", "os.system(f\"git clone {github_url}\")" ] }, { "cell_type": "markdown", "id": "047debfa-2c1e-4f76-bb11-4893ecb3d1e8", "metadata": {}, "source": [ "#### Download from R package" ] }, { "cell_type": "code", "execution_count": 6, "id": "d81f2c9d-362f-43cb-ad52-012e28217164", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:00:19.990342Z", "iopub.status.busy": "2025-04-07T18:00:19.990222Z", "iopub.status.idle": "2025-04-07T18:00:19.992712Z", "shell.execute_reply": "2025-04-07T18:00:19.992443Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Writing download.r\n" ] } ], "source": [ "%%writefile download.r\n", "\n", "options(repos = c(CRAN = \"https://cloud.r-project.org/\"))\n", "\n", "myinput.list=readRDS('MammalianMethylationConsortium/UniversalPanMammalianClock/ClockParameters/mydata_GitHub.Rds')\n", "anage=myinput.list[[3]]\n", "anage=subset(anage,select=c(SpeciesLatinName,GestationTimeInYears, averagedMaturity.yrs,maxAge))\n", "anage$HighmaxAge=1.3*anage$maxAge\n", "anage$HighmaxAge[anage$SpeciesLatinName=='Homo sapiens']=anage$maxAge[anage$SpeciesLatinName=='Homo sapiens']\n", "anage$HighmaxAge[anage$SpeciesLatinName=='Mus musculus']=anage$maxAge[anage$SpeciesLatinName=='Mus musculus']\n", "write.csv(anage, \"species_annotation.csv\")" ] }, { "cell_type": "code", "execution_count": 7, "id": "1ce2bac8-dd33-46cb-a7b6-14a1d0976f05", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:00:19.994180Z", "iopub.status.busy": "2025-04-07T18:00:19.994056Z", "iopub.status.idle": "2025-04-07T18:00:20.630642Z", "shell.execute_reply": "2025-04-07T18:00:20.629430Z" } }, "outputs": [ { "data": { "text/plain": [ "0" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "os.system(\"Rscript download.r\")" ] }, { "cell_type": "markdown", "id": "5035b180-3d1b-4432-8ebe-b9c92bd93a7f", "metadata": {}, "source": [ "## Load features" ] }, { "cell_type": "markdown", "id": "15f4af76-b93c-438c-b57f-f129d6e9ec99", "metadata": {}, "source": [ "#### From CSV file" ] }, { "cell_type": "code", "execution_count": 8, "id": "8a3d5de6-6303-487a-8b4d-e6345792f7be", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:00:20.632552Z", "iopub.status.busy": "2025-04-07T18:00:20.632417Z", "iopub.status.idle": "2025-04-07T18:00:20.645915Z", "shell.execute_reply": "2025-04-07T18:00:20.645138Z" } }, "outputs": [], "source": [ "df = pd.read_csv('MammalianMethylationConsortium/UniversalPanMammalianClock/ClockParameters/tissue_specific_clock/UniversalBloodClock2_final.csv')\n", "df['feature'] = df['var']\n", "df['coefficient'] = df['beta']\n", "cpg_features = df['feature'][1:].tolist()\n", "\n", "anage_df = pd.read_csv('species_annotation.csv', index_col=0)\n", "anage_df = anage_df[~anage_df['HighmaxAge'].isna()]\n", "anage_df = anage_df[~anage_df['GestationTimeInYears'].isna()]\n", "anage_df = anage_df.reset_index().drop('index', axis=1)\n", "anage_df = anage_df.fillna(0)\n", "species_features = anage_df['SpeciesLatinName'].tolist()\n", "\n", "model.features = cpg_features + species_features" ] }, { "cell_type": "markdown", "id": "ee6d8fa0-4767-4c45-9717-eb1c95e2ddc0", "metadata": {}, "source": [ "## Load weights into base model" ] }, { "cell_type": "code", "execution_count": 9, "id": "e09b3463-4fd4-41b1-ac21-e63ddd223fe0", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:00:20.649844Z", "iopub.status.busy": "2025-04-07T18:00:20.649494Z", "iopub.status.idle": "2025-04-07T18:00:20.655224Z", "shell.execute_reply": "2025-04-07T18:00:20.654157Z" } }, "outputs": [], "source": [ "weights = torch.tensor(df['coefficient'][1:].tolist()).unsqueeze(0)\n", "intercept = torch.tensor([df['coefficient'][0]])" ] }, { "cell_type": "markdown", "id": "ad261636-5b00-4979-bb1d-67a851f7aa19", "metadata": {}, "source": [ "#### Linear model" ] }, { "cell_type": "code", "execution_count": 10, "id": "d7f43b99-26f2-4622-9a76-316712058877", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:00:20.659016Z", "iopub.status.busy": "2025-04-07T18:00:20.658669Z", "iopub.status.idle": "2025-04-07T18:00:20.662630Z", "shell.execute_reply": "2025-04-07T18:00:20.662237Z" } }, "outputs": [], "source": [ "base_model = pya.models.LinearModel(input_dim=len(model.features))\n", "\n", "base_model.linear.weight.data = weights.float()\n", "base_model.linear.bias.data = intercept.float()\n", "\n", "model.base_model = base_model" ] }, { "cell_type": "markdown", "id": "ad8b4c1d-9d57-48b7-9a30-bcfea7b747b1", "metadata": {}, "source": [ "## Load reference values" ] }, { "cell_type": "code", "execution_count": 11, "id": "ade0f4c9-2298-4fc3-bb72-d200907dd731", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:00:20.664353Z", "iopub.status.busy": "2025-04-07T18:00:20.664226Z", "iopub.status.idle": "2025-04-07T18:00:20.667663Z", "shell.execute_reply": "2025-04-07T18:00:20.666712Z" } }, "outputs": [], "source": [ "reference_list = np.array([0] * len(model.features))\n", "reference_list[len(cpg_features) + np.where(anage_df.SpeciesLatinName == 'Homo sapiens')[0][0]] = 0.5\n", "model.reference_values = reference_list" ] }, { "cell_type": "markdown", "id": "af3bcf7b-74a8-4d21-9ccb-4de0c2b0516b", "metadata": {}, "source": [ "## Load preprocess and postprocess objects" ] }, { "cell_type": "code", "execution_count": 12, "id": "7a22fb20-c605-424d-8efb-7620c2c0755c", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:00:20.670694Z", "iopub.status.busy": "2025-04-07T18:00:20.670521Z", "iopub.status.idle": "2025-04-07T18:00:20.672897Z", "shell.execute_reply": "2025-04-07T18:00:20.672525Z" } }, "outputs": [], "source": [ "model.preprocess_name = None\n", "model.preprocess_dependencies = None" ] }, { "cell_type": "code", "execution_count": 13, "id": "ff4a21cb-cf41-44dc-9ed1-95cf8aa15772", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:00:20.675491Z", "iopub.status.busy": "2025-04-07T18:00:20.675165Z", "iopub.status.idle": "2025-04-07T18:00:20.680689Z", "shell.execute_reply": "2025-04-07T18:00:20.679824Z" } }, "outputs": [], "source": [ "model.postprocess_name = 'mammalian2'\n", "anage_df = anage_df.loc[:, ['GestationTimeInYears','averagedMaturity.yrs', 'maxAge','HighmaxAge']]\n", "anage_array = np.array(anage_df)\n", "model.postprocess_dependencies = [anage_array]" ] }, { "cell_type": "markdown", "id": "86e3d6b1-e67e-4f3d-bd39-0ebec5726c3c", "metadata": {}, "source": [ "## Check all clock parameters" ] }, { "cell_type": "code", "execution_count": 14, "id": "2168355c-47d9-475d-b816-49f65e74887c", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:00:20.684173Z", "iopub.status.busy": "2025-04-07T18:00:20.683837Z", "iopub.status.idle": "2025-04-07T18:00:20.691071Z", "shell.execute_reply": "2025-04-07T18:00:20.690792Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "%==================================== Model Details ====================================%\n", "Model Attributes:\n", "\n", "training: True\n", "metadata: {'approved_by_author': '⌛',\n", " 'citation': 'Lu, A. T., et al. \"Universal DNA methylation age across '\n", " 'mammalian tissues.\" Nature aging 3.9 (2023): 1144-1166.',\n", " 'clock_name': 'mammalianblood2',\n", " 'data_type': 'methylation',\n", " 'doi': 'https://doi.org/10.1038/s43587-023-00462-6',\n", " 'notes': None,\n", " 'research_only': None,\n", " 'species': 'multi',\n", " 'version': None,\n", " 'year': 2023}\n", "reference_values: array([0, 0, 0, ..., 0, 0, 0])\n", "preprocess_name: None\n", "preprocess_dependencies: None\n", "postprocess_name: 'mammalian2'\n", "postprocess_dependencies: [array([[2.19178082e-02, 2.49315068e+00, 3.60000000e+01, 4.68000000e+01],\n", " [1.64383562e-02, 3.00000000e+00, 1.15000000e+01, 1.49500000e+01],\n", " [3.01369863e-02, 4.50000000e+00, 1.50000000e+01, 1.95000000e+01],\n", " ...,\n", " [5.60876712e-01, 3.05123288e+00, 2.30000000e+01, 2.99000000e+01],\n", " [9.87671233e-02, 1.01706849e+00, 3.50000000e+00, 4.55000000e+00],\n", " [3.15068493e-01, 4.58334000e-01, 2.70000000e+01, 3.51000000e+01]])]\n", "features: ['cg00005271', 'cg00114412', 'cg00227334', 'cg00258262', 'cg00264615', 'cg00370791', 'cg00393367', 'cg00439117', 'cg00458823', 'cg00559067', 'cg00587168', 'cg00756693', 'cg00855890', 'cg00915004', 'cg00953859', 'cg00988788', 'cg01053290', 'cg01137681', 'cg01153166', 'cg01156745', 'cg01342051', 'cg01393939', 'cg01454752', 'cg01468874', 'cg01486146', 'cg01528792', 'cg01585174', 'cg01639959', 'cg01701526', 'cg01787104']... [Total elements: 2257]\n", "base_model_features: None\n", "\n", "%==================================== Model Details ====================================%\n", "Model Structure:\n", "\n", "base_model: LinearModel(\n", " (linear): Linear(in_features=2257, out_features=1, bias=True)\n", ")\n", "\n", "%==================================== Model Details ====================================%\n", "Model Parameters and Weights:\n", "\n", "base_model.linear.weight: [0.08963627368211746, -0.001239112694747746, -0.00347901857458055, 0.009207980707287788, -7.227044989122078e-05, -0.055550578981637955, 0.04472345858812332, -0.36105695366859436, -0.018508171662688255, 0.07516830414533615, 0.011418496258556843, 0.008542953990399837, 0.10099010914564133, 0.26528534293174744, 0.013976222835481167, -0.01059863530099392, -0.05601461976766586, 0.12440702319145203, 0.0005542261060327291, -0.06668984144926071, 0.03973907604813576, 0.08093400299549103, 0.018934927880764008, 0.002540371846407652, 0.07929009199142456, -0.14058975875377655, -0.01051307376474142, -0.11011394113302231, -0.06954656541347504, -0.06167547032237053]... [Tensor of shape torch.Size([1, 501])]\n", "base_model.linear.bias: tensor([1.0883])\n", "\n", "%==================================== Model Details ====================================%\n", "\n" ] } ], "source": [ "pya.utils.print_model_details(model)" ] }, { "cell_type": "markdown", "id": "986d0262-e0c7-4036-b687-dee53ba392fb", "metadata": {}, "source": [ "## Basic test" ] }, { "cell_type": "code", "execution_count": 15, "id": "936b9877-d076-4ced-99aa-e8d4c58c5caf", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:00:20.692532Z", "iopub.status.busy": "2025-04-07T18:00:20.692433Z", "iopub.status.idle": "2025-04-07T18:00:20.697724Z", "shell.execute_reply": "2025-04-07T18:00:20.697432Z" } }, "outputs": [ { "data": { "text/plain": [ "tensor([[ 3.8852],\n", " [ 0.5774],\n", " [-0.1589],\n", " [11.6877],\n", " [-0.1233],\n", " [ 5.4960],\n", " [-0.0658],\n", " [-0.5753],\n", " [10.2756],\n", " [42.2815]], dtype=torch.float64, grad_fn=)" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch.manual_seed(42)\n", "input = torch.randn(10, len(model.features), dtype=float)\n", "model.eval()\n", "model.to(float)\n", "pred = model(input)\n", "pred" ] }, { "cell_type": "markdown", "id": "fe8299d7-9285-4e22-82fd-b664434b4369", "metadata": {}, "source": [ "## Save torch model" ] }, { "cell_type": "code", "execution_count": 16, "id": "5ef2fa8d-c80b-4fdd-8555-79c0d541788e", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:00:20.699129Z", "iopub.status.busy": "2025-04-07T18:00:20.699034Z", "iopub.status.idle": "2025-04-07T18:00:20.703405Z", "shell.execute_reply": "2025-04-07T18:00:20.703066Z" } }, "outputs": [], "source": [ "torch.save(model, f\"../weights/{model.metadata['clock_name']}.pt\")" ] }, { "cell_type": "markdown", "id": "bac6257b-8d08-4a90-8d0b-7f745dc11ac1", "metadata": {}, "source": [ "## Clear directory\n", "" ] }, { "cell_type": "code", "execution_count": 17, "id": "11aeaa70-44c0-42f9-86d7-740e3849a7a6", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:00:20.705096Z", "iopub.status.busy": "2025-04-07T18:00:20.704976Z", "iopub.status.idle": "2025-04-07T18:00:20.854911Z", "shell.execute_reply": "2025-04-07T18:00:20.854275Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Deleted file: coefficients.xlsx\n", "Deleted file: species_annotation.csv\n", "Deleted folder: MammalianMethylationConsortium\n", "Deleted file: download.r\n" ] } ], "source": [ "# Function to remove a folder and all its contents\n", "def remove_folder(path):\n", " try:\n", " shutil.rmtree(path)\n", " print(f\"Deleted folder: {path}\")\n", " except Exception as e:\n", " print(f\"Error deleting folder {path}: {e}\")\n", "\n", "# Get a list of all files and folders in the current directory\n", "all_items = os.listdir('.')\n", "\n", "# Loop through the items\n", "for item in all_items:\n", " # Check if it's a file and does not end with .ipynb\n", " if os.path.isfile(item) and not item.endswith('.ipynb'):\n", " os.remove(item)\n", " print(f\"Deleted file: {item}\")\n", " # Check if it's a folder\n", " elif os.path.isdir(item):\n", " remove_folder(item)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.7" } }, "nbformat": 4, "nbformat_minor": 5 }