{ "cells": [ { "cell_type": "markdown", "id": "b5dc2c08-1f4a-4e75-be2a-ea319edab2f1", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "# Supervised Pixel Classifiers\n", "\n", "This notebook depends on the main pipeline image analysis steps having been done first!" ] }, { "cell_type": "code", "execution_count": 1, "id": "eef5f57b-2d81-435f-b625-c8c7da39c3e5", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The PalmettoBUG package is copyrighted 2024-2025 by the Medical University of South Carolina and licensed under the GPL-3 license.\n", "It is free & open source software, can be redistributed in compliance with the GPL3 license, and comes with absolutely no warranty.\n", "In python, use palmettobug.print_license() to see the license, or use palmettobug.print_3rd_party_license_info() to print information\n", "about the licenses and copyright of 3rd party software used in PalmettoBUG itself or in the creation of PalmettoBUG.\n" ] } ], "source": [ "import os\n", "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "import seaborn.objects as so\n", "\n", "import tifffile as tf\n", "\n", "import palmettobug as pbug" ] }, { "cell_type": "code", "execution_count": 2, "id": "30608c3b-9223-41c8-b071-ce53445074ef", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'0.2.11'" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "pbug.__version__" ] }, { "cell_type": "markdown", "id": "eec748c3-e750-4ec1-8664-cf1d2bb06bad", "metadata": {}, "source": [ "### CHANGE The following directory to match an existing directory on your computer if you are testing this tutorial on your own machine!" ] }, { "cell_type": "code", "execution_count": 3, "id": "528b2dae-6fe5-4880-9344-31c1a741b932", "metadata": {}, "outputs": [], "source": [ "my_computer_path = \"C:/Users/Default/Desktop\" ## CHANGE This DIRECTORY to match an existing directory on your computer if you testing this tutorial on your own machine!" ] }, { "cell_type": "markdown", "id": "1ae7350c-c5ea-4724-b964-51f03cc300ca", "metadata": {}, "source": [ "## Setup a Supervised Pixel Classifier\n", "\n", "\n", "A variety of parameters must be decided when setting up a supervised classifier:\n", "\n", " 1). The classifier name\n", " \n", " 2). directories\n", " a. the directory where the pixel classifier will be setup in (it will contain the classifier information, output, etc.)\n", " b. the directory of images from which the classificaiton will be run\n", " \n", " 3). The number of classes to be predicted (and ideally, a dictionary matching each class # with a biological label).\n", " - The first class is always 'background', and gets treated differently by default\n", " - The class dictionary is optional at first (as training, prediction, etc. can proceed without it), but will be needed later\n", " if you want meaningful labels to be assigned to each class. Otherwise, the classes will only be represented as numbers.\n", " \n", " 4). A dictionary of the channels to use in the classification\n", " - only channels in the dictionary will be used!\n", " - format of dictionary = {'Channel_name': channel_integer}\n", " - The key part for successful training / prediction is the integer (not the channel name), as whatever\n", " integers are provided as values in this dictionary will be the channels of the image used in classification.\n", " The channel names are useful because they are saved in a .json file and provide a record of what antigens\n", " were intended to be used. However, the channel names are not used in training / prediction itself.\n", "\n", " 5). Features to use from the channels\n", " - These are transformations of the channel data that occur before they are provided to the classifier\n", " - For example, the 'GAUSSIAN' feature is the simplest available feature and involves a gaussian blurring\n", " of the channels before being supplied to the classifier. NOTE: there is no 'no transformation' feature and the \n", " untransformed values of the channels are not passed into the classifier -- because of this, the 'GAUSSIAN' feature\n", " is the feature that represents the intensities of the channel and is essentially always used. \n", " - Other features can bring out details, such as edges, gradients, etc. in the channels' signal, but do not represent simple\n", " channel intensity. For a full examination of what these features are, see QuPath's documentation about pixel classification\n", " as it is more thorough & QuPath is where the features offered in palmettobug are derived from.\n", " \n", " 6). The Sigma(s) for blurring channels\n", " - These determine how much blurring is applied before / while deriving features for a channel. Smaller sigmas mean smaller blurring kernels, which in\n", " turn means more locally determined values for the features, whereas larger sigmas means larger blurring kernels and more contribution of non-local / distant \n", " pixels to each channel's features.\n", " - More than one sigma can be used at once, which can be useful in order to provide the classifier information about different scales in the image.\n", " - The final number of information layers passed to the classifier is equal to (the number of sigmas) * (the number of channels) * (the number of features)\n", " This is because all possible combinations of sigmas and features are applied to every selected channel. This contrasts with Unsupervised Classifiers.\n", "\n", "Other, less commonly needed parameters -- such as training rate, internal architecture for the classifier, or whether to return categorial classification maps -- can also be set at this step\n", "but can often be ignored. Of these, the most likely to edit would be the internal architecture of the neural network used as a classifier. This is supplied as a list of integers that \n", "determine how many neurons to place in the hidden layers of the network. By default, the classifier's network has no hidden layers." ] }, { "cell_type": "code", "execution_count": 4, "id": "0bea4ef6-e4b0-4ec3-820f-57458fd5e726", "metadata": {}, "outputs": [], "source": [ "''' name and directories '''\n", "\n", "my_classifier_name = \"lumen_epithelia_laminapropria.json\"\n", "\n", "pixel_directory = f\"{my_computer_path}/Example_IMC\"\n", "pixel_class_object = pbug.SupervisedClassifier(homedir = pixel_directory) ## the pixel classifier object can be created with only the set-up directory,\n", " ## but before training can be proceed we will need to devided more parameters\n", "\n", "image_directory = pixel_directory + \"/images/img\" ## the source of the images you want to classify train / predict from" ] }, { "cell_type": "code", "execution_count": 5, "id": "2f499b95-ff89-4705-88b6-d6753be3c12e", "metadata": {}, "outputs": [], "source": [ "''' Select classes & setup dictionary '''\n", "\n", "classes = [\"background\", \"epithelia\", \"lamina_propria\"] ## \"background\" should always be the first class -- in this case it wil also be the stand-in for 'lumen'\n", "classes_dictionary = {1:\"background\",2:\"epithelia\",3:\"lamina_propria\"} ### when the classifier exports a classification prediction, and when training labels are made in Napari, it will be in the form of integer labels\n", " ## this dictionary identifies the correspondence betwen integer labels, and the biological labels\n", " ## WARNING! 'background' should ALWAYS BE THE FIRST CLASS! This is because class 1 is automatically treated differently by palmettobug\n", "number_of_classes = len(classes)" ] }, { "cell_type": "code", "execution_count": 6, "id": "a75b7e76-cff5-420b-ae39-5e0c59c67b55", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'aSMA': 0,\n", " 'p-selectin': 1,\n", " 'Vimentin': 2,\n", " 'CD14': 3,\n", " 'CD31': 4,\n", " 'CD16': 5,\n", " 'Pan-Keratin': 6,\n", " 'CD11b': 7,\n", " 'CD163': 8,\n", " 'CD45': 9,\n", " 'CD206': 10,\n", " 'CCL2': 11,\n", " 'FoxP3': 12,\n", " 'CD4': 13,\n", " 'E-cadherin': 14,\n", " 'CD68': 15,\n", " 'CD66b': 16,\n", " 'CD20': 17,\n", " 'CD8': 18,\n", " 'Vitrionectin': 19,\n", " 'CD32b': 20,\n", " 'GranzymeB': 21,\n", " 'Ki-67': 22,\n", " 'Collagen-1': 23,\n", " 'CD3': 24,\n", " 'HistoneH3': 25,\n", " 'Beta-Catenin': 26,\n", " 'CD45RO': 27,\n", " 'HLA-DR': 28,\n", " 'DNA1': 29,\n", " 'DNA2': 30,\n", " 'Seg1': 31,\n", " 'Seg2': 32,\n", " 'Seg3': 33}" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "''' Channel Dictionary \n", "\n", "Here I select all channels using the panel + a loop, but otherwise this should be set manually with the format as in the following example:\n", " {'Vimentin': 2, 'Vitronectin': 19, 'Beta-Catenin': 26}\n", "\n", "Where the number for each channel corresponds to that channel's order in the images (zero-indexed / as it would be displayed in napari), or \n", "according to that channel's index in the panel file (after dropping unwanted channels & reseting the index)\n", "'''\n", "panel = pd.read_csv(f\"{pixel_directory}/panel.csv\")\n", "panel = panel[panel['keep'] == 1].reset_index() ########## drop & reset index to use the panel file!\n", "\n", "channel_dictionary = {} \n", "for i,ii in zip(panel.index, panel['name']):\n", " channel_dictionary[ii] = i\n", "\n", "channel_dictionary" ] }, { "cell_type": "code", "execution_count": 7, "id": "519a4673-92ef-42f7-8ecc-2de684441a51", "metadata": {}, "outputs": [], "source": [ "''' Selects sigma(s) and feature(s) '''\n", "\n", "sigma_list = [1.0, 2.0, 4.0] ## a list of floats, denotes the sigmas to use for features --> see QuPath supervised pixel classifiers for details of what this means\n", "\n", "possible_features = [\"GAUSSIAN\", \"LAPLACIAN\", \"WEIGHTED_STD_DEV\", \"GRADIENT_MAGNITUDE\", \"STRUCTURE_TENSOR_EIGENVALUE_MAX\", \n", " \"STRUCTURE_TENSOR_EIGENVALUE_MIN\", \"STRUCTURE_TENSOR_COHERENCE\", \"HESSIAN_DETERMINANT\", \n", " \"HESSIAN_EIGENVALUE_MAX\", \"HESSIAN_EIGENVALUE_MIN\"]\n", "\n", "features_list = [\"GAUSSIAN\"] ## fewer features is often better, see QuPath pixel classification documentation for details. Each choesn feature is applied to ALL channels\n", " # GAUSSIAN is almost always recommended (it is the blurred representation of the selected channels)" ] }, { "cell_type": "code", "execution_count": 8, "id": "e0781bf3-41cd-4c00-9a53-f32247e2be9d", "metadata": {}, "outputs": [], "source": [ "internal_architecture = [] ### or a list of integers indicating te sizes of hidden layers in the ANN_MLP. Such as [16,8]. \n", "\n", "## this function writes the dictionary / .json file to the classifier directory and sets up the ANN_MLP for training:\n", "_ = pixel_class_object.setup_classifier(classifier_name = my_classifier_name, number_of_classes = number_of_classes, \n", " sigma_list = sigma_list, features_list = features_list, \n", " channel_dictionary = channel_dictionary, image_directory = image_directory, classes_dictionary = classes_dictionary, \n", " categorical = True, internal_architecture = internal_architecture, epsilon = 0.01, iterations = 1000)" ] }, { "cell_type": "markdown", "id": "8a21243b-78d2-49f5-81d8-120195d5bfb8", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "## Creating Training labels\n", "\n", "Once the classifier is setup, we must manually label parts of our images as each of the classes (including background) in the classifier.\n", "This can be conveniently done inside Napari -- although that means that this tutorial cannot directly show how that is done, as the labeling\n", "steps happens outside the notebook.\n", "\n", "Here, we launch one of source images in napari, along with a label layer that we can annotate inside nappari to represent our training labels.\n", "Once the labels are completed, we can close napari and run the second cell that writes our labels to a folder inside the classifier's directory." ] }, { "cell_type": "code", "execution_count": 11, "id": "06ba6154-95b3-4a7f-bba9-69d530125da1", "metadata": {}, "outputs": [], "source": [ "'''NOTE! The training labels used in this example notebook can be found in the /training_labels subfolder where this notebook is!\n", "\n", "If you want to exactly reproduce the results of these notebooks on your machine, move that labeling image into your pixel classifier by uncommenting the 4 lines below,\n", "and then be sure to not edit the labeling in the following two cells (running those two cells can still be useful to see the labeling layer, but ANY edits will ruin reproducibility of the classifier):\n", "'''\n", "\n", "import shutil\n", "shutil.rmtree(pixel_class_object.classifier_training_labels)\n", "shutil.copytree(os.getcwd() + \"/training_labels\", pixel_class_object.classifier_training_labels) ## be sure your current working directory is where this notebook is saved!\n", "\n", "display()\n", "\n", "############### Only run this cell once!" ] }, { "cell_type": "code", "execution_count": 12, "id": "e1a297af-1dc1-4c48-8a09-124813c0e35d", "metadata": {}, "outputs": [], "source": [ "'''\n", "Make training labels (done inside napari, this code only launches napari) \n", "\n", "Repeat this code as many times as desired for the various images in your dataset, until you have the number of annotated images that you want\n", "\n", "Remember your class dictionary pairings: {1:\"background\", 2:\"epithelia\", 3:\"lamina_propria\"} while making labels!\n", "\n", "If you have previously made a set of training labels for an image, and then re-open that image using this code, napari should automatically\n", "load your prior labels, allowing you examine, update, and edit those prior labels. \n", "'''\n", "image_number = 0\n", "\n", "image_paths =[\"\".join([image_directory,\"/\",i]) for i in os.listdir(image_directory)]\n", "\n", "import warnings\n", "warnings.filterwarnings(\"ignore\", message = \"pyside_type_init\")\n", "pixel_class_object.launch_Napari_px(image_path = image_paths[image_number], display_all_channels = False) ### display_all_channels changes the axis of the napari display" ] }, { "cell_type": "code", "execution_count": 13, "id": "82fbb2ec-499b-45b2-b1ec-25dfb99df70e", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Training labels written!\n" ] } ], "source": [ "pixel_class_object.write_from_Napari() ## this function is necessary to write the labels made in napari after napari has been closed. --> it must be in a separate cell\n", "\n", "### WARNING! this needs to be run afterwards in a separate cell as the Napari launch!" ] }, { "cell_type": "markdown", "id": "22c168c2-8db0-4844-894e-11368b46a2d3", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "## Training & Prediction\n", "\n", "Once training labels exist, training and prediction are extremely simple, only requiring that you pass in the source directory for images (which should be the \n", "same as the image directory you had previously chosen).\n", "\n", "During training, all training labels in the classifer's training_labels directory are used -- This means that if you want to discard previously saved traning labels,\n", "you must edit them in napari, or manually delete them from the classifier's training_labels directory. \n", "\n", "NOTE!: Normally, you would want to make labels for multiple different images, creating training lables from diverse images and diverse areas of the images. \n", "Doing this is important to help improve the classifier's performance across your dataset -- otherwise it may only be competent at classifying pixels from images that look like the image(s) \n", "you made training labels for. However, because this is just demonstration code, though, I will not bother with making multiple rounds of training labels." ] }, { "cell_type": "code", "execution_count": 14, "id": "6edbf3dc-38cb-456a-84f9-fc8a51b98055", "metadata": {}, "outputs": [], "source": [ "'''\n", "Now, Train & Predict with the classifier\n", "'''\n", "_ = pixel_class_object.train_folder(image_directory)" ] }, { "cell_type": "code", "execution_count": 15, "id": "7dff5e92-cad2-4834-ae0e-6b831b0125ea", "metadata": {}, "outputs": [], "source": [ "'''\n", "Now generate pixel classification maps from our training classifier:\n", "'''\n", "pixel_class_object.predict_folder(image_directory) ## if specificied, the output_directory for the predictions can be set to a differnet location than the default folder inside the classifier's directory" ] }, { "cell_type": "code", "execution_count": 16, "id": "1284e319-301a-4c57-b176-00f44b587b9c", "metadata": {}, "outputs": [], "source": [ "'''Now, let's look at one of our predictions:'''\n", "\n", "prediction_paths = [\"\".join([pixel_class_object.output_directory,\"/\",i]) for i in os.listdir(pixel_class_object.output_directory)] \n", "\n", "image_number = 7 ## image 0 looks decent because that is the only image where I made my training labels, but for image 7 (and others) the pixel class predictions look quite poor! \n", " ## If your predictions look poor, then make addiitonal training labels for the cases where the classifier had been failing, then repeat training & prediciton\n", "\n", "pbug.run_napari(image = tf.imread(image_paths[image_number]), masks = tf.imread(prediction_paths[image_number]))" ] }, { "cell_type": "markdown", "id": "1d10c75f-472a-416b-989a-9a884d4a49a2", "metadata": {}, "source": [ "## What can you do with these Classifier Predictions?" ] }, { "cell_type": "markdown", "id": "637a4a32-b2b9-4fee-bfd9-ebe1f792d904", "metadata": {}, "source": [ "## 1). crop / slice images to mimum area encompassing the class of interest\n", "\n", "You can crop images down to only be the regions where you class of interest is\n", "Or: set background / undesirable / noisy pixel to zero" ] }, { "cell_type": "code", "execution_count": 17, "id": "8369bc81-48ef-435c-a8ed-e0ae060ab8c0", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Key not found: list index out of range\n", "Key not found: list index out of range\n", "key not found list index out of range\n", "Key not found: list index out of range\n", "Key not found: list index out of range\n", "Key not found: list index out of range\n", "key not found list index out of range\n", "Key not found: list index out of range\n", "Key not found: list index out of range\n", "Key not found: list index out of range\n", "key not found list index out of range\n", "Key not found: list index out of range\n", "Key not found: list index out of range\n", "Key not found: list index out of range\n", "key not found list index out of range\n", "Key not found: list index out of range\n", "Key not found: list index out of range\n", "Key not found: list index out of range\n", "key not found list index out of range\n", "Key not found: list index out of range\n", "Key not found: list index out of range\n", "Key not found: list index out of range\n", "key not found list index out of range\n", "Key not found: list index out of range\n", "Key not found: list index out of range\n", "Key not found: list index out of range\n", "key not found list index out of range\n", "Key not found: list index out of range\n", "Key not found: list index out of range\n", "Key not found: list index out of range\n", "key not found list index out of range\n", "Key not found: list index out of range\n", "Key not found: list index out of range\n", "Key not found: list index out of range\n", "key not found list index out of range\n", "Key not found: list index out of range\n", "Key not found: list index out of range\n", "Key not found: list index out of range\n", "key not found list index out of range\n", "Key not found: list index out of range\n" ] } ], "source": [ "class_to_keep = [2] ## remmber that class 2 was epithelia\n", "class_map_folder = pixel_class_object.output_directory\n", "image_folder = image_directory\n", "output_folder = pixel_directory + \"/images/sliced_by_epithelia\"\n", "padding = 0 ## whether to keep a layer of pixels around the class\n", "zero_out = False ## whether to additionally set all channels in the non-class pixels to 0\n", "\n", "\n", "pbug.slice_folder(class_to_keep = class_to_keep,\n", " class_map_folder = class_map_folder, \n", " image_folder = image_folder, \n", " output_folder = output_folder,\n", " padding = padding, zero_out = zero_out)" ] }, { "cell_type": "markdown", "id": "915b24b0-d83a-4392-87b5-b389bd93e919", "metadata": {}, "source": [ "## 2). Directly Segment objects using the classifier\n", "\n", "This generally depends on the objects you want to segment being of similar size, being round / convex, and not overlapping too much.\n", "For example, this style of segmentation can be effective for nuclei." ] }, { "cell_type": "code", "execution_count": 18, "id": "311f32f7-1aa4-400d-a4e1-f67d83cbf2c4", "metadata": {}, "outputs": [], "source": [ "'''\n", "Direct segmentation from pixel classifier:\n", "The class that is used for segmentation is split into masks by calculating distance-from-background, finding the maximum points, and using those points as seeds for a watershedding algorithm\n", "This requires that 'cells' / masking regions be well-separated and have distinct break points between cells\n", "\n", "A more usual use case might be to classify nuclei from background to get nuclei masks -- which tend to be round and more easily separable by this method --\n", "then apply an expansion (whether by merging with a pixel classifier's output, or by using a simple/fixed pixel expansion) to try to capture the cytoplasm around the nuclei.\n", "'''\n", "pixel_classifier_directory = pixel_class_object.output_directory\n", "output_folder = pixel_directory + \"/masks/lumen_segmentation\"\n", "threshold = 25 ### masks smaller that this threshold will ber prohibited\n", "distance_between_centroids = 45 ## changing this parameter determines how close the centroids of the masks can be during watershedding, for increasing it favors larger, fewer masks while decreasing it favors many smaller masks.\n", " ## Increasing it can be very useful to curb excessive fragmentation of masks\n", "background = 3\n", "'''Here we 'segment' the lumens of crypts in the image. Why? Becuase of the objects in this particular classifier, the lumen are the most circular, and therefore will likely be the best behaved when segmenting this way.'''\n", "to_segment_on = [1] ## a list, by integer, of the classes to include in the segmentation (in this case, GFAP / astrocyte)\n", "\n", "pbug.segment_class_map_folder(pixel_classifier_directory = pixel_classifier_directory, \n", " output_folder = output_folder, \n", " distance_between_centroids = distance_between_centroids,\n", " threshold = threshold, \n", " to_segment_on = to_segment_on, \n", " background = background) ## as currently set up, this only work on the final class in the images, after background is removed / zero'd out)" ] }, { "cell_type": "markdown", "id": "6ab0002e-ce53-431e-a169-0a6b7bdbb5dd", "metadata": {}, "source": [ "## 3). Classify segmented cells using the clasifier\n", "\n", "We can use a classifer to cluster cells into groups, by looking at what pixel classes are inside the segmenttaion regions.\n", "Here, we will classify the cells as belonging to either lumen, epithelia, or lamina_propia by looking at what is the most common pixel class \n", "within each cell (as in, the mode). " ] }, { "cell_type": "code", "execution_count": 19, "id": "48ec08ec-82f6-4128-b6d9-8e2414de19bc", "metadata": {}, "outputs": [], "source": [ "'''\n", "Now, classify the deepcell masks using the classification maps we just generated\n", "'''\n", "name = \"My_classy_deepcell_masks\"\n", "\n", "mesmer_mask_folder = pixel_directory + \"/masks/example_deepcell_masks\" ## or \"/masks/deepcell\" \n", "classifier_mask_folder = pixel_class_object.output_directory\n", "run_folder = pixel_directory + f\"/classy_masks/{name}\"\n", "output_folder = run_folder + f\"/{name}\" \n", "\n", "if not os.path.exists(run_folder):\n", " os.mkdir(run_folder)\n", "\n", "mask_classifications = pbug.mode_classify_folder(mesmer_mask_folder, classifier_mask_folder, output_folder, merging_table = None)\n", "\n", "## to save the mask_classifications for a potential later import into a CATALYST-style analysis:\n", "mask_classifications.to_csv(run_folder + f\"/{name}.csv\", index = False) \n", " ## This precise naming is chosen for compatibility with loading into the GUI later -- however if you never intend\n", " ## to use the GUI, then any name for this will do." ] }, { "cell_type": "code", "execution_count": 20, "id": "e3e8511b-9511-43d4-a990-7c795bc7d46b", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
classification
03
12
23
32
42
......
46653
46663
46673
46683
46693
\n", "

36927 rows × 1 columns

\n", "
" ], "text/plain": [ " classification\n", "0 3\n", "1 2\n", "2 3\n", "3 2\n", "4 2\n", "... ...\n", "4665 3\n", "4666 3\n", "4667 3\n", "4668 3\n", "4669 3\n", "\n", "[36927 rows x 1 columns]" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Unnamed: 0classlabelsmerging
001background0
112epithelia2
223lamina_propria3
\n", "
" ], "text/plain": [ " Unnamed: 0 class labels merging\n", "0 0 1 background 0\n", "1 1 2 epithelia 2\n", "2 2 3 lamina_propria 3" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "'''The classifications are stored as numbers, but these can decoded back into biological labels using the 'biological_labels' csv in the pixel classifier.'''\n", "\n", "display(mask_classifications)\n", "pd.read_csv(pixel_class_object.classifier_dir + \"/biological_labels.csv\")" ] }, { "cell_type": "markdown", "id": "aa6ba9e4-667c-43ea-a6cb-d9829b65e093", "metadata": {}, "source": [ "## 4). Extend masks into matching pixel classification\n", "\n", "Existing cell segmentation masks can be expanded by pixel class. This can be particularly useful in cases where the segmentation algorithms\n", "are mainly finidng circular, simple shapes, but the expected biological cell shapes are highly irregular / branched. \n", "Example: astrocyte segmentation can benefit from making an astrocyte/GFAP pixel class, classifying cells as astrocyte from that, then \n", " extending those astrocyte cells into the surrounding astrocyte pixel class -- \"decorating\" the circular astorcyte cell bodies\n", " identified by cell segmentation with the irregular cytoplasmic projections captured by the pixel classifier.\n", "\n", "Specifically:\n", "\n", " First, you MUST have an existing cell segmentation and have classified that set of cell segmentation masks (see use #3 above). \n", " \"Classy masks\" are necessary so that cell not of the pixel class of interest will not be extended.\n", " \n", " Second, you must select which classes to extend masks into. Often, this is a very specific class (for example, GFAP pixels for astrocytes), and usually\n", " you don't want to extend into every class (especially background), as this tends to extend cells into massive, non-biological shapes. For example, \n", " in this case extending cells into the 'lumen class' will mean that the rare cells in the 'lumen' regions (which we expect to be largely empty / acellular) \n", " could be expanded to fill the entire 'lumen' region that they inside of.\n", " \n", " Next, inside the mask extending function the classes of interest are iterated over, with each cell mask belonging to the class being expanded by\n", " watershedding (from the centroid of the cell mask) into any pixels of that class that touch the mask. The watershedding is done simultaneously\n", " for all masks of a given class at once, so that final boundaries for multiple cells matching cells in contact with a pixel class region is determined \n", " by the watershedding (the original cell masks protected from losing pixels to a neighboring cell in this process)." ] }, { "cell_type": "code", "execution_count": 21, "id": "46057f12-3e3f-482c-8edb-5f2ca16fb3da", "metadata": {}, "outputs": [], "source": [ "'''\n", "Now merge the masks and pixel classifier together, with the help of the classy masks!\n", "'''\n", "classy_mesmer_mask_folder = output_folder ## from classy masksing above\n", "output_directory_folder = pixel_directory + \"/masks/extended_masks\"\n", "\n", "connectivity = 2 ## 1 or 2, an argument in skimage's watershedding function -- determines if diagonally adjacent pixels are considered adjacent for the watershedding\n", "merge_list = [2,3] ## the classes to merge --> in this case epithelia and lamina_propria\n", "\n", "pbug.extend_masks_folder(classifier_mask_folder, mesmer_mask_folder, \n", " classy_mesmer_mask_folder, output_directory_folder,\n", " merge_list = merge_list, \n", " connectivity = connectivity)" ] }, { "cell_type": "code", "execution_count": null, "id": "87ed79b2-7e2c-4c55-b5a8-f6fb5739f8fa", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.16" } }, "nbformat": 4, "nbformat_minor": 5 }