{ "cells": [ { "cell_type": "markdown", "id": "2f04eee0-5928-4e74-a754-6dc2e528810c", "metadata": {}, "source": [ "# MammalianSkin2" ] }, { "cell_type": "markdown", "id": "a3f514a3-772c-4a14-afdf-5a8376851ff4", "metadata": {}, "source": [ "## Index\n", "1. [Instantiate model class](#Instantiate-model-class)\n", "2. [Define clock metadata](#Define-clock-metadata)\n", "3. [Download clock dependencies](#Download-clock-dependencies)\n", "5. [Load features](#Load-features)\n", "6. [Load weights into base model](#Load-weights-into-base-model)\n", "7. [Load reference values](#Load-reference-values)\n", "8. [Load preprocess and postprocess objects](#Load-preprocess-and-postprocess-objects)\n", "10. [Check all clock parameters](#Check-all-clock-parameters)\n", "10. [Basic test](#Basic-test)\n", "11. [Save torch model](#Save-torch-model)\n", "12. [Clear directory](#Clear-directory)" ] }, { "cell_type": "markdown", "id": "d95fafdc-643a-40ea-a689-200bd132e90c", "metadata": {}, "source": [ "Let's first import some packages:" ] }, { "cell_type": "code", "execution_count": 1, "id": "4adfb4de-cd79-4913-a1af-9e23e9e236c9", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:02:50.740548Z", "iopub.status.busy": "2025-04-07T18:02:50.740103Z", "iopub.status.idle": "2025-04-07T18:02:52.185101Z", "shell.execute_reply": "2025-04-07T18:02:52.184782Z" } }, "outputs": [], "source": [ "import os\n", "import inspect\n", "import shutil\n", "import json\n", "import torch\n", "import pandas as pd\n", "import pyaging as pya\n", "import numpy as np" ] }, { "cell_type": "markdown", "id": "145082e5-ced4-47ae-88c0-cb69773e3c5a", "metadata": {}, "source": [ "## Instantiate model class" ] }, { "cell_type": "code", "execution_count": 2, "id": "8aa77372-7ed3-4da7-abc9-d30372106139", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:02:52.186854Z", "iopub.status.busy": "2025-04-07T18:02:52.186635Z", "iopub.status.idle": "2025-04-07T18:02:52.195059Z", "shell.execute_reply": "2025-04-07T18:02:52.194797Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "class MammalianSkin2(pyagingModel):\n", " def __init__(self):\n", " super().__init__()\n", "\n", " def forward(self, x):\n", " x_cpg = x[:, :-1756] # number of species in lookup table\n", " x_species = x[:, -1756:] # number of species in lookup table\n", " x = self.base_model(x_cpg)\n", " x = self.postprocess(x, x_species)\n", " return x\n", "\n", " def preprocess(self, x):\n", " return x\n", "\n", " def postprocess(self, x, x_species):\n", " \"\"\"\n", " Converts output of relative age to age in units of years.\n", " \"\"\"\n", " indices = torch.argmax(x_species, dim=1)\n", " anage_array = self.postprocess_dependencies[0]\n", " anage_tensor = torch.tensor(anage_array, dtype=x.dtype, device=x.device)\n", " gestation_time = anage_tensor[indices, 0].unsqueeze(1)\n", " max_age = anage_tensor[indices, 3].unsqueeze(1)\n", "\n", " x = torch.exp(-torch.exp(-x))\n", " x = x * (max_age + gestation_time) - gestation_time\n", " return x\n", "\n" ] } ], "source": [ "def print_entire_class(cls):\n", " source = inspect.getsource(cls)\n", " print(source)\n", "\n", "print_entire_class(pya.models.MammalianSkin2)" ] }, { "cell_type": "code", "execution_count": 3, "id": "78536494-f1d9-44de-8583-c89a310d2307", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:02:52.196449Z", "iopub.status.busy": "2025-04-07T18:02:52.196354Z", "iopub.status.idle": "2025-04-07T18:02:52.198015Z", "shell.execute_reply": "2025-04-07T18:02:52.197779Z" } }, "outputs": [], "source": [ "model = pya.models.MammalianSkin2()" ] }, { "cell_type": "markdown", "id": "51f8615e-01fa-4aa5-b196-3ee2b35d261c", "metadata": {}, "source": [ "## Define clock metadata" ] }, { "cell_type": "code", "execution_count": 4, "id": "6601da9e-8adc-44ee-9308-75e3cd31b816", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:02:52.199333Z", "iopub.status.busy": "2025-04-07T18:02:52.199254Z", "iopub.status.idle": "2025-04-07T18:02:52.201245Z", "shell.execute_reply": "2025-04-07T18:02:52.200987Z" } }, "outputs": [], "source": [ "model.metadata[\"clock_name\"] = \"mammalianskin2\"\n", "model.metadata[\"data_type\"] = \"DNA methylation\" # Paper: methylation\n", "model.metadata[\"species\"] = \"multiple species\" # Paper: multi\n", "model.metadata[\"year\"] = 2023\n", "model.metadata[\"approved_by_author\"] = \"⌛\"\n", "model.metadata[\"citation\"] = \"Lu, A. T., et al. \\\"Universal DNA methylation age across mammalian tissues.\\\" Nature Aging 3.9 (2023): 1144-1166.\"\n", "model.metadata[\"doi\"] = \"https://doi.org/10.1038/s43587-023-00462-6\"\n", "model.metadata[\"notes\"] = \"Skin-specific universal clock 2 fits relative age and returns species-adjusted chronological age in years after the maximum-lifespan inverse transformation.\"\n", "model.metadata[\"research_only\"] = None\n", "model.metadata[\"tissue\"] = [\"skin\"] # Paper: skin\n", "model.metadata[\"predicts\"] = [\"chronological age\"] # Paper: The official inverse transformation returns DNAm chronological age in years.\n", "model.metadata[\"training_target\"] = [\"relative age\"] # Paper: relative age\n", "model.metadata[\"unit\"] = [\"years\"] # Paper: The official inverse transformation returns DNAm chronological age in years.\n", "model.metadata[\"model_type\"] = \"elastic net regression\" # Paper: Elastic net\n", "model.metadata[\"platform\"] = [\"Horvath MammalMethylChip40\"] # Paper: HorvathMammalMethylChip40\n", "model.metadata[\"population\"] = \"multiple mammalian species\" # Paper: multi-species mammalian skin samples\n", "model.metadata[\"journal\"] = \"Nature Aging\"\n", "model.metadata[\"last_author\"] = \"Steve Horvath\"\n", "model.metadata[\"n_features\"] = 2240\n", "model.metadata[\"citations\"] = 390\n", "model.metadata[\"citations_date\"] = \"2026-07-05\"\n" ] }, { "cell_type": "markdown", "id": "74492239-5aae-4026-9d90-6bc9c574c110", "metadata": {}, "source": [ "## Download clock dependencies" ] }, { "cell_type": "markdown", "id": "5881f33e-0486-45d1-bd87-9c995f47ca62", "metadata": {}, "source": [ "#### Download GitHub repository" ] }, { "cell_type": "code", "execution_count": 5, "id": "e85525f7-6da4-4962-a7d5-0607e76eea33", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:02:52.202566Z", "iopub.status.busy": "2025-04-07T18:02:52.202485Z", "iopub.status.idle": "2025-04-07T18:05:07.436534Z", "shell.execute_reply": "2025-04-07T18:05:07.436184Z" } }, "outputs": [ { "data": { "text/plain": [ "0" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "github_url = \"https://github.com/shorvath/MammalianMethylationConsortium.git\"\n", "github_folder_name = github_url.split('/')[-1].split('.')[0]\n", "os.system(f\"git clone {github_url}\")" ] }, { "cell_type": "markdown", "id": "047debfa-2c1e-4f76-bb11-4893ecb3d1e8", "metadata": {}, "source": [ "#### Download from R package" ] }, { "cell_type": "code", "execution_count": 6, "id": "d81f2c9d-362f-43cb-ad52-012e28217164", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:05:07.438075Z", "iopub.status.busy": "2025-04-07T18:05:07.437971Z", "iopub.status.idle": "2025-04-07T18:05:07.440427Z", "shell.execute_reply": "2025-04-07T18:05:07.440140Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Writing download.r\n" ] } ], "source": [ "%%writefile download.r\n", "\n", "options(repos = c(CRAN = \"https://cloud.r-project.org/\"))\n", "\n", "myinput.list=readRDS('MammalianMethylationConsortium/UniversalPanMammalianClock/ClockParameters/mydata_GitHub.Rds')\n", "anage=myinput.list[[3]]\n", "anage=subset(anage,select=c(SpeciesLatinName,GestationTimeInYears, averagedMaturity.yrs,maxAge))\n", "anage$HighmaxAge=1.3*anage$maxAge\n", "anage$HighmaxAge[anage$SpeciesLatinName=='Homo sapiens']=anage$maxAge[anage$SpeciesLatinName=='Homo sapiens']\n", "anage$HighmaxAge[anage$SpeciesLatinName=='Mus musculus']=anage$maxAge[anage$SpeciesLatinName=='Mus musculus']\n", "write.csv(anage, \"species_annotation.csv\")" ] }, { "cell_type": "code", "execution_count": 7, "id": "1ce2bac8-dd33-46cb-a7b6-14a1d0976f05", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:05:07.441592Z", "iopub.status.busy": "2025-04-07T18:05:07.441492Z", "iopub.status.idle": "2025-04-07T18:05:08.062677Z", "shell.execute_reply": "2025-04-07T18:05:08.062365Z" } }, "outputs": [ { "data": { "text/plain": [ "0" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "os.system(\"Rscript download.r\")" ] }, { "cell_type": "markdown", "id": "5035b180-3d1b-4432-8ebe-b9c92bd93a7f", "metadata": {}, "source": [ "## Load features" ] }, { "cell_type": "markdown", "id": "15f4af76-b93c-438c-b57f-f129d6e9ec99", "metadata": {}, "source": [ "#### From CSV file" ] }, { "cell_type": "code", "execution_count": 8, "id": "8a3d5de6-6303-487a-8b4d-e6345792f7be", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:05:08.064186Z", "iopub.status.busy": "2025-04-07T18:05:08.064072Z", "iopub.status.idle": "2025-04-07T18:05:08.074790Z", "shell.execute_reply": "2025-04-07T18:05:08.074460Z" } }, "outputs": [], "source": [ "df = pd.read_csv('MammalianMethylationConsortium/UniversalPanMammalianClock/ClockParameters/tissue_specific_clock/UniversalSkinClock2_final.csv')\n", "df['feature'] = df['var']\n", "df['coefficient'] = df['beta']\n", "cpg_features = df['feature'][1:].tolist()\n", "\n", "anage_df = pd.read_csv('species_annotation.csv', index_col=0)\n", "anage_df = anage_df[~anage_df['HighmaxAge'].isna()]\n", "anage_df = anage_df[~anage_df['GestationTimeInYears'].isna()]\n", "anage_df = anage_df.reset_index().drop('index', axis=1)\n", "anage_df = anage_df.fillna(0)\n", "species_features = anage_df['SpeciesLatinName'].tolist()\n", "\n", "model.features = cpg_features + species_features" ] }, { "cell_type": "markdown", "id": "ee6d8fa0-4767-4c45-9717-eb1c95e2ddc0", "metadata": {}, "source": [ "## Load weights into base model" ] }, { "cell_type": "code", "execution_count": 9, "id": "e09b3463-4fd4-41b1-ac21-e63ddd223fe0", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:05:08.077193Z", "iopub.status.busy": "2025-04-07T18:05:08.077023Z", "iopub.status.idle": "2025-04-07T18:05:08.079431Z", "shell.execute_reply": "2025-04-07T18:05:08.079152Z" } }, "outputs": [], "source": [ "weights = torch.tensor(df['coefficient'][1:].tolist()).unsqueeze(0)\n", "intercept = torch.tensor([df['coefficient'][0]])" ] }, { "cell_type": "markdown", "id": "ad261636-5b00-4979-bb1d-67a851f7aa19", "metadata": {}, "source": [ "#### Linear model" ] }, { "cell_type": "code", "execution_count": 10, "id": "d7f43b99-26f2-4622-9a76-316712058877", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:05:08.080880Z", "iopub.status.busy": "2025-04-07T18:05:08.080766Z", "iopub.status.idle": "2025-04-07T18:05:08.083194Z", "shell.execute_reply": "2025-04-07T18:05:08.082863Z" } }, "outputs": [], "source": [ "base_model = pya.models.LinearModel(input_dim=len(model.features))\n", "\n", "base_model.linear.weight.data = weights.float()\n", "base_model.linear.bias.data = intercept.float()\n", "\n", "model.base_model = base_model" ] }, { "cell_type": "markdown", "id": "ad8b4c1d-9d57-48b7-9a30-bcfea7b747b1", "metadata": {}, "source": [ "## Load reference values" ] }, { "cell_type": "code", "execution_count": 11, "id": "ade0f4c9-2298-4fc3-bb72-d200907dd731", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:05:08.085347Z", "iopub.status.busy": "2025-04-07T18:05:08.085208Z", "iopub.status.idle": "2025-04-07T18:05:08.087733Z", "shell.execute_reply": "2025-04-07T18:05:08.087417Z" } }, "outputs": [], "source": [ "reference_list = np.array([0] * len(model.features))\n", "reference_list[len(cpg_features) + np.where(anage_df.SpeciesLatinName == 'Homo sapiens')[0][0]] = 0.5\n", "model.reference_values = reference_list" ] }, { "cell_type": "markdown", "id": "af3bcf7b-74a8-4d21-9ccb-4de0c2b0516b", "metadata": {}, "source": [ "## Load preprocess and postprocess objects" ] }, { "cell_type": "code", "execution_count": 12, "id": "7a22fb20-c605-424d-8efb-7620c2c0755c", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:05:08.089348Z", "iopub.status.busy": "2025-04-07T18:05:08.089218Z", "iopub.status.idle": "2025-04-07T18:05:08.090938Z", "shell.execute_reply": "2025-04-07T18:05:08.090643Z" } }, "outputs": [], "source": [ "model.preprocess_name = None\n", "model.preprocess_dependencies = None" ] }, { "cell_type": "code", "execution_count": 13, "id": "ff4a21cb-cf41-44dc-9ed1-95cf8aa15772", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:05:08.092267Z", "iopub.status.busy": "2025-04-07T18:05:08.092177Z", "iopub.status.idle": "2025-04-07T18:05:08.095033Z", "shell.execute_reply": "2025-04-07T18:05:08.094240Z" } }, "outputs": [], "source": [ "model.postprocess_name = 'mammalian2'\n", "anage_df = anage_df.loc[:, ['GestationTimeInYears','averagedMaturity.yrs', 'maxAge',\t'HighmaxAge']]\n", "anage_array = np.array(anage_df)\n", "model.postprocess_dependencies = [anage_array]" ] }, { "cell_type": "markdown", "id": "86e3d6b1-e67e-4f3d-bd39-0ebec5726c3c", "metadata": {}, "source": [ "## Check all clock parameters" ] }, { "cell_type": "code", "execution_count": 14, "id": "2168355c-47d9-475d-b816-49f65e74887c", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:05:08.096650Z", "iopub.status.busy": "2025-04-07T18:05:08.096522Z", "iopub.status.idle": "2025-04-07T18:05:08.100952Z", "shell.execute_reply": "2025-04-07T18:05:08.100634Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "%==================================== Model Details ====================================%\n", "Model Attributes:\n", "\n", "training: True\n", "metadata: {'approved_by_author': '⌛',\n", " 'citation': 'Lu, A. T., et al. \"Universal DNA methylation age across '\n", " 'mammalian tissues.\" Nature aging 3.9 (2023): 1144-1166.',\n", " 'clock_name': 'mammalianskin2',\n", " 'data_type': 'methylation',\n", " 'doi': 'https://doi.org/10.1038/s43587-023-00462-6',\n", " 'notes': None,\n", " 'research_only': None,\n", " 'species': 'multi',\n", " 'version': None,\n", " 'year': 2023}\n", "reference_values: array([0, 0, 0, ..., 0, 0, 0])\n", "preprocess_name: None\n", "preprocess_dependencies: None\n", "postprocess_name: 'mammalian2'\n", "postprocess_dependencies: [array([[2.19178082e-02, 2.49315068e+00, 3.60000000e+01, 4.68000000e+01],\n", " [1.64383562e-02, 3.00000000e+00, 1.15000000e+01, 1.49500000e+01],\n", " [3.01369863e-02, 4.50000000e+00, 1.50000000e+01, 1.95000000e+01],\n", " ...,\n", " [5.60876712e-01, 3.05123288e+00, 2.30000000e+01, 2.99000000e+01],\n", " [9.87671233e-02, 1.01706849e+00, 3.50000000e+00, 4.55000000e+00],\n", " [3.15068493e-01, 4.58334000e-01, 2.70000000e+01, 3.51000000e+01]])]\n", "features: ['cg00006213', 'cg00012521', 'cg00038376', 'cg00106940', 'cg00138195', 'cg00147638', 'cg00149213', 'cg00153046', 'cg00200794', 'cg00207040', 'cg00247020', 'cg00281640', 'cg00384342', 'cg00386499', 'cg00422680', 'cg00437193', 'cg00438946', 'cg00573041', 'cg00595237', 'cg00619575', 'cg00623160', 'cg00662491', 'cg00689651', 'cg00694357', 'cg00823476', 'cg00917676', 'cg00973544', 'cg01019040', 'cg01052727', 'cg01134518']... [Total elements: 2240]\n", "base_model_features: None\n", "\n", "%==================================== Model Details ====================================%\n", "Model Structure:\n", "\n", "base_model: LinearModel(\n", " (linear): Linear(in_features=2240, out_features=1, bias=True)\n", ")\n", "\n", "%==================================== Model Details ====================================%\n", "Model Parameters and Weights:\n", "\n", "base_model.linear.weight: [0.10117671638727188, -0.014433339238166809, 0.01582280360162258, -0.16444504261016846, 0.2054484486579895, 0.0524430125951767, -0.001450160751119256, -0.043094977736473083, 0.11684814095497131, 0.044266875833272934, -0.4065758287906647, 0.12912878394126892, -0.08310002088546753, -0.24308426678180695, -0.03736037760972977, -0.03317352384328842, -0.027269257232546806, 0.16627272963523865, 0.0917305126786232, -1.0780762434005737, 0.008392410352826118, 0.008044207468628883, 0.0011250120587646961, -0.02045692503452301, -0.16058066487312317, 0.02744181826710701, -0.09590887278318405, -1.1931358575820923, -0.025111977010965347, 0.05292962118983269]... [Tensor of shape torch.Size([1, 484])]\n", "base_model.linear.bias: tensor([-1.8751])\n", "\n", "%==================================== Model Details ====================================%\n", "\n" ] } ], "source": [ "pya.utils.print_model_details(model)" ] }, { "cell_type": "markdown", "id": "986d0262-e0c7-4036-b687-dee53ba392fb", "metadata": {}, "source": [ "## Basic test" ] }, { "cell_type": "code", "execution_count": 15, "id": "936b9877-d076-4ced-99aa-e8d4c58c5caf", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:05:08.102501Z", "iopub.status.busy": "2025-04-07T18:05:08.102401Z", "iopub.status.idle": "2025-04-07T18:05:08.107846Z", "shell.execute_reply": "2025-04-07T18:05:08.107522Z" } }, "outputs": [ { "data": { "text/plain": [ "tensor([[-3.6164e-01],\n", " [-1.0493e+00],\n", " [ 2.7006e+01],\n", " [-5.7418e-02],\n", " [ 8.0249e+01],\n", " [ 2.4591e+01],\n", " [-8.7671e-02],\n", " [ 1.5445e+01],\n", " [-3.2877e-02],\n", " [ 1.2089e+00]], dtype=torch.float64, grad_fn=)" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch.manual_seed(42)\n", "input = torch.randn(10, len(model.features), dtype=float)\n", "model.eval()\n", "model.to(float)\n", "pred = model(input)\n", "pred" ] }, { "cell_type": "markdown", "id": "fe8299d7-9285-4e22-82fd-b664434b4369", "metadata": {}, "source": [ "## Save torch model" ] }, { "cell_type": "code", "execution_count": 16, "id": "5ef2fa8d-c80b-4fdd-8555-79c0d541788e", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:05:08.109575Z", "iopub.status.busy": "2025-04-07T18:05:08.109447Z", "iopub.status.idle": "2025-04-07T18:05:08.113103Z", "shell.execute_reply": "2025-04-07T18:05:08.112754Z" } }, "outputs": [], "source": [ "torch.save(model, f\"../weights/{model.metadata['clock_name']}.pt\")" ] }, { "cell_type": "markdown", "id": "bac6257b-8d08-4a90-8d0b-7f745dc11ac1", "metadata": {}, "source": [ "## Clear directory\n", "" ] }, { "cell_type": "code", "execution_count": 17, "id": "11aeaa70-44c0-42f9-86d7-740e3849a7a6", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T18:05:08.114586Z", "iopub.status.busy": "2025-04-07T18:05:08.114471Z", "iopub.status.idle": "2025-04-07T18:05:08.262406Z", "shell.execute_reply": "2025-04-07T18:05:08.261324Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Deleted file: species_annotation.csv\n", "Deleted folder: MammalianMethylationConsortium\n", "Deleted file: download.r\n" ] } ], "source": [ "# Function to remove a folder and all its contents\n", "def remove_folder(path):\n", " try:\n", " shutil.rmtree(path)\n", " print(f\"Deleted folder: {path}\")\n", " except Exception as e:\n", " print(f\"Error deleting folder {path}: {e}\")\n", "\n", "# Get a list of all files and folders in the current directory\n", "all_items = os.listdir('.')\n", "\n", "# Loop through the items\n", "for item in all_items:\n", " # Check if it's a file and does not end with .ipynb\n", " if os.path.isfile(item) and not item.endswith('.ipynb'):\n", " os.remove(item)\n", " print(f\"Deleted file: {item}\")\n", " # Check if it's a folder\n", " elif os.path.isdir(item):\n", " remove_folder(item)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.7" } }, "nbformat": 4, "nbformat_minor": 5 }