{ "cells": [ { "cell_type": "markdown", "id": "2f04eee0-5928-4e74-a754-6dc2e528810c", "metadata": {}, "source": [ "# Horvath2013" ] }, { "cell_type": "markdown", "id": "a3f514a3-772c-4a14-afdf-5a8376851ff4", "metadata": {}, "source": [ "## Index\n", "1. [Instantiate model class](#Instantiate-model-class)\n", "2. [Define clock metadata](#Define-clock-metadata)\n", "3. [Download clock dependencies](#Download-clock-dependencies)\n", "5. [Load features](#Load-features)\n", "6. [Load weights into base model](#Load-weights-into-base-model)\n", "7. [Load reference values](#Load-reference-values)\n", "8. [Load preprocess and postprocess objects](#Load-preprocess-and-postprocess-objects)\n", "10. [Check all clock parameters](#Check-all-clock-parameters)\n", "10. [Basic test](#Basic-test)\n", "11. [Save torch model](#Save-torch-model)\n", "12. [Clear directory](#Clear-directory)" ] }, { "cell_type": "markdown", "id": "d95fafdc-643a-40ea-a689-200bd132e90c", "metadata": {}, "source": [ "Let's first import some packages:" ] }, { "cell_type": "code", "execution_count": 1, "id": "4adfb4de-cd79-4913-a1af-9e23e9e236c9", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:53.979350Z", "iopub.status.busy": "2025-04-07T17:51:53.979155Z", "iopub.status.idle": "2025-04-07T17:51:55.337257Z", "shell.execute_reply": "2025-04-07T17:51:55.336910Z" } }, "outputs": [], "source": [ "import os\n", "import inspect\n", "import shutil\n", "import json\n", "import torch\n", "import pandas as pd\n", "import pyaging as pya" ] }, { "cell_type": "markdown", "id": "145082e5-ced4-47ae-88c0-cb69773e3c5a", "metadata": {}, "source": [ "## Instantiate model class" ] }, { "cell_type": "code", "execution_count": 2, "id": "8aa77372-7ed3-4da7-abc9-d30372106139", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:55.338955Z", "iopub.status.busy": "2025-04-07T17:51:55.338738Z", "iopub.status.idle": "2025-04-07T17:51:55.346182Z", "shell.execute_reply": "2025-04-07T17:51:55.345901Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "class Horvath2013(pyagingModel):\n", " def __init__(self):\n", " super().__init__()\n", "\n", " def preprocess(self, x):\n", " return x\n", "\n", " def postprocess(self, x):\n", " \"\"\"\n", " Applies an anti-logarithmic linear transformation to a PyTorch tensor.\n", " \"\"\"\n", " adult_age = 20\n", "\n", " # Create a mask for negative and non-negative values\n", " mask_negative = x < 0\n", " mask_non_negative = ~mask_negative\n", "\n", " # Initialize the result tensor\n", " age_tensor = torch.empty_like(x)\n", "\n", " # Exponential transformation for negative values\n", " age_tensor[mask_negative] = (1 + adult_age) * torch.exp(x[mask_negative]) - 1\n", "\n", " # Linear transformation for non-negative values\n", " age_tensor[mask_non_negative] = (1 + adult_age) * x[mask_non_negative] + adult_age\n", "\n", " return age_tensor\n", "\n" ] } ], "source": [ "def print_entire_class(cls):\n", " source = inspect.getsource(cls)\n", " print(source)\n", "\n", "print_entire_class(pya.models.Horvath2013)" ] }, { "cell_type": "code", "execution_count": 3, "id": "78536494-f1d9-44de-8583-c89a310d2307", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:55.347464Z", "iopub.status.busy": "2025-04-07T17:51:55.347373Z", "iopub.status.idle": "2025-04-07T17:51:55.349004Z", "shell.execute_reply": "2025-04-07T17:51:55.348766Z" } }, "outputs": [], "source": [ "model = pya.models.Horvath2013()" ] }, { "cell_type": "markdown", "id": "51f8615e-01fa-4aa5-b196-3ee2b35d261c", "metadata": {}, "source": [ "## Define clock metadata" ] }, { "cell_type": "code", "execution_count": 4, "id": "6601da9e-8adc-44ee-9308-75e3cd31b816", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:55.350282Z", "iopub.status.busy": "2025-04-07T17:51:55.350197Z", "iopub.status.idle": "2025-04-07T17:51:55.352160Z", "shell.execute_reply": "2025-04-07T17:51:55.351928Z" } }, "outputs": [], "source": [ "model.metadata[\"clock_name\"] = \"horvath2013\"\n", "model.metadata[\"data_type\"] = \"DNA methylation\" # Paper: The predictor uses CpG DNA-methylation levels.\n", "model.metadata[\"species\"] = \"Homo sapiens\" # Paper: The predictor was trained on human tissues and cell types.\n", "model.metadata[\"year\"] = 2013\n", "model.metadata[\"approved_by_author\"] = \"⌛\"\n", "model.metadata[\"citation\"] = \"Horvath, S. DNA methylation age of human tissues and cell types. Genome Biology 14, R115 (2013).\"\n", "model.metadata[\"doi\"] = \"https://doi.org/10.1186/gb-2013-14-10-r115\"\n", "model.metadata[\"notes\"] = \"Pan-tissue DNAm-age predictor fitted by elastic net to a transformed chronological-age outcome and returned to the year scale; it uses 353 CpGs shared between the 27K and 450K arrays.\"\n", "model.metadata[\"research_only\"] = None\n", "model.metadata[\"tissue\"] = [\"multi-tissue\"] # Paper: Training datasets represented a wide spectrum among 51 healthy tissues and cell types.\n", "model.metadata[\"predicts\"] = [\"chronological age\"] # Paper: The model returns DNA-methylation age calibrated to chronological age.\n", "model.metadata[\"training_target\"] = [\"chronological age\"] # Paper: A transformed version of chronological age was the elastic-net outcome.\n", "model.metadata[\"unit\"] = [\"years\"] # Paper: DNAm age and prediction error are reported in years after inverse transformation.\n", "model.metadata[\"model_type\"] = \"elastic net regression\" # Paper: The paper identifies the penalized regression model as elastic net.\n", "model.metadata[\"platform\"] = [\"Illumina 27K\", \"Illumina 450K\"] # Paper: Training used CpGs shared by the Illumina 27K and 450K platforms.\n", "model.metadata[\"population\"] = \"all ages\" # Paper: The training and validation collection spans many tissues and ages, including newborn and centenarian-range samples.\n", "model.metadata[\"journal\"] = \"Genome Biology\"\n", "model.metadata[\"last_author\"] = \"Steve Horvath\"\n", "model.metadata[\"n_features\"] = 353\n", "model.metadata[\"citations\"] = 7318\n", "model.metadata[\"citations_date\"] = \"2026-07-05\"\n" ] }, { "cell_type": "markdown", "id": "74492239-5aae-4026-9d90-6bc9c574c110", "metadata": {}, "source": [ "## Download clock dependencies" ] }, { "cell_type": "markdown", "id": "9a5b163b-1b25-4c2f-ad0c-86f0f2ed39d0", "metadata": {}, "source": [ "#### Download directly with curl" ] }, { "cell_type": "code", "execution_count": 5, "id": "28fdcad4-1f62-4da7-b556-1ecc8cf3d0e0", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:55.353438Z", "iopub.status.busy": "2025-04-07T17:51:55.353359Z", "iopub.status.idle": "2025-04-07T17:51:55.634472Z", "shell.execute_reply": "2025-04-07T17:51:55.633635Z" } }, "outputs": [ { "data": { "text/plain": [ "0" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "supplementary_url = \"https://static-content.springer.com/esm/art%3A10.1186%2Fgb-2013-14-10-r115/MediaObjects/13059_2013_3156_MOESM3_ESM.csv\"\n", "supplementary_file_name = \"coefficients.csv\"\n", "os.system(f\"curl -o {supplementary_file_name} {supplementary_url}\")" ] }, { "cell_type": "code", "execution_count": 6, "id": "86accc98-8ad9-4ca8-9a53-dbec3c32e816", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:55.637819Z", "iopub.status.busy": "2025-04-07T17:51:55.637526Z", "iopub.status.idle": "2025-04-07T17:51:56.021415Z", "shell.execute_reply": "2025-04-07T17:51:56.020455Z" } }, "outputs": [ { "data": { "text/plain": [ "0" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "supplementary_url = \"https://static-content.springer.com/esm/art%3A10.1186%2Fgb-2013-14-10-r115/MediaObjects/13059_2013_3156_MOESM22_ESM.csv\"\n", "supplementary_file_name = \"reference_feature_values.csv\"\n", "os.system(f\"curl -o {supplementary_file_name} {supplementary_url}\")" ] }, { "cell_type": "markdown", "id": "5035b180-3d1b-4432-8ebe-b9c92bd93a7f", "metadata": {}, "source": [ "## Load features" ] }, { "cell_type": "markdown", "id": "15f4af76-b93c-438c-b57f-f129d6e9ec99", "metadata": {}, "source": [ "#### From CSV file" ] }, { "cell_type": "code", "execution_count": 7, "id": "8a3d5de6-6303-487a-8b4d-e6345792f7be", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:56.026340Z", "iopub.status.busy": "2025-04-07T17:51:56.025990Z", "iopub.status.idle": "2025-04-07T17:51:56.038097Z", "shell.execute_reply": "2025-04-07T17:51:56.037463Z" } }, "outputs": [], "source": [ "df = pd.read_csv('coefficients.csv', skiprows=2)\n", "df['feature'] = df['CpGmarker']\n", "df['coefficient'] = df['CoefficientTraining']\n", "model.features = df['feature'][1:].tolist()" ] }, { "cell_type": "markdown", "id": "ee6d8fa0-4767-4c45-9717-eb1c95e2ddc0", "metadata": {}, "source": [ "## Load weights into base model" ] }, { "cell_type": "code", "execution_count": 8, "id": "e09b3463-4fd4-41b1-ac21-e63ddd223fe0", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:56.040927Z", "iopub.status.busy": "2025-04-07T17:51:56.040703Z", "iopub.status.idle": "2025-04-07T17:51:56.044360Z", "shell.execute_reply": "2025-04-07T17:51:56.043798Z" } }, "outputs": [], "source": [ "weights = torch.tensor(df['coefficient'][1:].tolist()).unsqueeze(0)\n", "intercept = torch.tensor([df['coefficient'][0]])" ] }, { "cell_type": "markdown", "id": "ad261636-5b00-4979-bb1d-67a851f7aa19", "metadata": {}, "source": [ "#### Linear model" ] }, { "cell_type": "code", "execution_count": 9, "id": "d7f43b99-26f2-4622-9a76-316712058877", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:56.046690Z", "iopub.status.busy": "2025-04-07T17:51:56.046525Z", "iopub.status.idle": "2025-04-07T17:51:56.050147Z", "shell.execute_reply": "2025-04-07T17:51:56.049637Z" } }, "outputs": [], "source": [ "base_model = pya.models.LinearModel(input_dim=len(model.features))\n", "\n", "base_model.linear.weight.data = weights.float()\n", "base_model.linear.bias.data = intercept.float()\n", "\n", "model.base_model = base_model" ] }, { "cell_type": "markdown", "id": "ad8b4c1d-9d57-48b7-9a30-bcfea7b747b1", "metadata": {}, "source": [ "## Load reference values" ] }, { "cell_type": "markdown", "id": "f7fdae64-096a-4640-ade7-6a17b78a01d5", "metadata": {}, "source": [ "#### From CSV file" ] }, { "cell_type": "code", "execution_count": 10, "id": "86de757f-fb38-4bcb-b91e-fc3372d22aad", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:56.052658Z", "iopub.status.busy": "2025-04-07T17:51:56.052471Z", "iopub.status.idle": "2025-04-07T17:51:56.075773Z", "shell.execute_reply": "2025-04-07T17:51:56.075318Z" } }, "outputs": [], "source": [ "reference_feature_values_df = pd.read_csv('reference_feature_values.csv', index_col=0)\n", "reference_feature_values_df = reference_feature_values_df.loc[model.features]\n", "model.reference_values = reference_feature_values_df['goldstandard2'].tolist()" ] }, { "cell_type": "markdown", "id": "af3bcf7b-74a8-4d21-9ccb-4de0c2b0516b", "metadata": {}, "source": [ "## Load preprocess and postprocess objects" ] }, { "cell_type": "code", "execution_count": 11, "id": "7a22fb20-c605-424d-8efb-7620c2c0755c", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:56.077812Z", "iopub.status.busy": "2025-04-07T17:51:56.077659Z", "iopub.status.idle": "2025-04-07T17:51:56.079552Z", "shell.execute_reply": "2025-04-07T17:51:56.079224Z" } }, "outputs": [], "source": [ "model.preprocess_name = None\n", "model.preprocess_dependencies = None" ] }, { "cell_type": "code", "execution_count": 12, "id": "ff4a21cb-cf41-44dc-9ed1-95cf8aa15772", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:56.081113Z", "iopub.status.busy": "2025-04-07T17:51:56.080983Z", "iopub.status.idle": "2025-04-07T17:51:56.082811Z", "shell.execute_reply": "2025-04-07T17:51:56.082450Z" } }, "outputs": [], "source": [ "model.postprocess_name = 'anti_log_linear'\n", "model.postprocess_dependencies = None" ] }, { "cell_type": "markdown", "id": "86e3d6b1-e67e-4f3d-bd39-0ebec5726c3c", "metadata": {}, "source": [ "## Check all clock parameters" ] }, { "cell_type": "code", "execution_count": 13, "id": "2168355c-47d9-475d-b816-49f65e74887c", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:56.084524Z", "iopub.status.busy": "2025-04-07T17:51:56.084394Z", "iopub.status.idle": "2025-04-07T17:51:56.088597Z", "shell.execute_reply": "2025-04-07T17:51:56.088298Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "%==================================== Model Details ====================================%\n", "Model Attributes:\n", "\n", "training: True\n", "metadata: {'approved_by_author': '⌛',\n", " 'citation': 'Horvath, Steve. \"DNA methylation age of human tissues and cell '\n", " 'types.\" Genome biology 14.10 (2013): 1-20.',\n", " 'clock_name': 'horvath2013',\n", " 'data_type': 'methylation',\n", " 'doi': 'https://doi.org/10.1186/gb-2013-14-10-r115',\n", " 'notes': None,\n", " 'research_only': None,\n", " 'species': 'Homo sapiens',\n", " 'version': None,\n", " 'year': 2013}\n", "reference_values: [0.790221397, 0.89001929, 0.059106387, 0.23651937, 0.073668777, 0.563295909, 0.864999404, 0.027047887, 0.660721193, 0.033420176, 0.047913033, 0.517283973, 0.050756537, 0.072267723, 0.014877693, 0.876157036, 0.187082703, 0.170303406, 0.019217238, 0.560726569, 0.845964086, 0.447995921, 0.055406903, 0.059821557, 0.533869814, 0.065190933, 0.896227421, 0.090932158, 0.032431793, 0.480007151]... [Total elements: 353]\n", "preprocess_name: None\n", "preprocess_dependencies: None\n", "postprocess_name: 'anti_log_linear'\n", "postprocess_dependencies: None\n", "features: ['cg00075967', 'cg00374717', 'cg00864867', 'cg00945507', 'cg01027739', 'cg01353448', 'cg01584473', 'cg01644850', 'cg01656216', 'cg01873645', 'cg01968178', 'cg02085507', 'cg02154074', 'cg02217159', 'cg02331561', 'cg02332492', 'cg02364642', 'cg02388150', 'cg02479575', 'cg02489552', 'cg02580606', 'cg02654291', 'cg02827112', 'cg02972551', 'cg03103192', 'cg03167275', 'cg03270204', 'cg03565323', 'cg03588357', 'cg03760483']... [Total elements: 353]\n", "base_model_features: None\n", "\n", "%==================================== Model Details ====================================%\n", "Model Structure:\n", "\n", "base_model: LinearModel(\n", " (linear): Linear(in_features=353, out_features=1, bias=True)\n", ")\n", "\n", "%==================================== Model Details ====================================%\n", "Model Parameters and Weights:\n", "\n", "base_model.linear.weight: [0.12933661043643951, 0.005017857067286968, 1.599764108657837, 0.056852418929338455, 0.10286285728216171, 0.23856045305728912, 0.08862839639186859, 0.1599487066268921, 0.04280552640557289, -0.6040563583374023, 1.1692458391189575, 0.006127551198005676, 0.04475700482726097, -0.08651260286569595, 0.12692885100841522, 0.06277134269475937, -0.024270255118608475, 0.42597126960754395, 1.8754462003707886, 0.0737413838505745, 0.34927448630332947, 0.14058348536491394, 0.05064608156681061, 0.4739460051059723, 0.02353634312748909, -0.5510412454605103, 0.013544725254178047, -0.2076532244682312, -0.8588430285453796, 0.014946399256587029]... [Tensor of shape torch.Size([1, 353])]\n", "base_model.linear.bias: tensor([0.6955])\n", "\n", "%==================================== Model Details ====================================%\n", "\n" ] } ], "source": [ "pya.utils.print_model_details(model)" ] }, { "cell_type": "markdown", "id": "986d0262-e0c7-4036-b687-dee53ba392fb", "metadata": {}, "source": [ "## Basic test" ] }, { "cell_type": "code", "execution_count": 14, "id": "936b9877-d076-4ced-99aa-e8d4c58c5caf", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:56.090196Z", "iopub.status.busy": "2025-04-07T17:51:56.090093Z", "iopub.status.idle": "2025-04-07T17:51:56.094564Z", "shell.execute_reply": "2025-04-07T17:51:56.094250Z" } }, "outputs": [ { "data": { "text/plain": [ "tensor([[ -0.9718],\n", " [ 1.8615],\n", " [ -0.9980],\n", " [ 67.1194],\n", " [253.6647],\n", " [ 61.3025],\n", " [ -0.9928],\n", " [ -0.6078],\n", " [ -0.9995],\n", " [ -0.7832]], dtype=torch.float64, grad_fn=)" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch.manual_seed(42)\n", "input = torch.randn(10, len(model.features), dtype=float)\n", "model.eval()\n", "model.to(float)\n", "pred = model(input)\n", "pred" ] }, { "cell_type": "markdown", "id": "fe8299d7-9285-4e22-82fd-b664434b4369", "metadata": {}, "source": [ "## Save torch model" ] }, { "cell_type": "code", "execution_count": 15, "id": "5ef2fa8d-c80b-4fdd-8555-79c0d541788e", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:56.096031Z", "iopub.status.busy": "2025-04-07T17:51:56.095930Z", "iopub.status.idle": "2025-04-07T17:51:56.100301Z", "shell.execute_reply": "2025-04-07T17:51:56.099998Z" } }, "outputs": [], "source": [ "torch.save(model, f\"../weights/{model.metadata['clock_name']}.pt\")" ] }, { "cell_type": "markdown", "id": "bac6257b-8d08-4a90-8d0b-7f745dc11ac1", "metadata": {}, "source": [ "## Clear directory\n", "" ] }, { "cell_type": "code", "execution_count": 16, "id": "11aeaa70-44c0-42f9-86d7-740e3849a7a6", "metadata": { "execution": { "iopub.execute_input": "2025-04-07T17:51:56.101705Z", "iopub.status.busy": "2025-04-07T17:51:56.101603Z", "iopub.status.idle": "2025-04-07T17:51:56.105358Z", "shell.execute_reply": "2025-04-07T17:51:56.105059Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Deleted file: coefficients.csv\n", "Deleted file: reference_feature_values.csv\n", "Deleted file: coefficients.xlsx\n" ] } ], "source": [ "# Function to remove a folder and all its contents\n", "def remove_folder(path):\n", " try:\n", " shutil.rmtree(path)\n", " print(f\"Deleted folder: {path}\")\n", " except Exception as e:\n", " print(f\"Error deleting folder {path}: {e}\")\n", "\n", "# Get a list of all files and folders in the current directory\n", "all_items = os.listdir('.')\n", "\n", "# Loop through the items\n", "for item in all_items:\n", " # Check if it's a file and does not end with .ipynb\n", " if os.path.isfile(item) and not item.endswith('.ipynb'):\n", " os.remove(item)\n", " print(f\"Deleted file: {item}\")\n", " # Check if it's a folder\n", " elif os.path.isdir(item):\n", " remove_folder(item)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.7" } }, "nbformat": 4, "nbformat_minor": 5 }