Analyzing Pixel Classification “As a Whole”

This notebook depends on the SupervisedClassifier notebook having been run first!!

import os
import numpy as np
import pandas as pd

import palmettobug as pbug

The PalmettoBUG package is copyrighted 2024-2025 by the Medical University of South Carolina and licensed under the GPL-3 license.
It is free & open source software, can  be redistributed in compliance with the GPL3 license, and comes with absolutely no warranty.
In python, use palmettobug.print_license() to see the license, or use palmettobug.print_3rd_party_license_info() to print information
about the licenses and copyright of 3rd party software used in PalmettoBUG itself or in the creation of PalmettoBUG.

pbug.__version__

'0.2.11'

CHANGE The following directory to match an existing directory on your computer if you are testing this tutorial on your own machine!

my_computer_path =  "C:/Users/Default/Desktop"  ## CHANGE This DIRECTORY to match an existing directory on your computer if you testing this tutorial on your own machine!

Setting up for a Whole-Class Analysis

This requires:

1). a pixel classification to have been run (usually a supervised classifier, but sometimes a merged / annotated unsupervised classifier could be used)

2). Region measurements must be taken from the classifier's predictions (treating them as if they are segmentation masks)

3). The set-up of Analysis_panel.csv and metadata.csv files, just like for standard single-cell / CATALYST-like analyses

In this case, requirements #1 and #3 will depend on other notebooks where those steps have already been done, but requirement #2 we do within this notebook, then load for whole-class analysis.

''' Gather directories from which we'll read pixel class region properties '''

project_directory = f'{my_computer_path}/Example_IMC'

pixel_classifier_output = f'{project_directory}/Pixel_Classification/lumen_epithelia_laminapropria/classification_maps'

whole_class_directory = f'{project_directory}/Pixel_Classification/lumen_epithelia_laminapropria/Whole_class_analysis'
if not os.path.exists(whole_class_directory):
    os.mkdir(whole_class_directory)

''' Make region measurements from pixel classifications '''

experiment = pbug.imc_entrypoint(project_directory, resolutions =  [1.0, 1.0], from_mcds = False)
analysis_folder_name = "Whole_class_analysis"
input_img_folder = experiment.directory_object.img_dir + "/img"
experiment.make_segmentation_measurements(input_img_folder,
                                          pixel_classifier_output,
                                          output_intensities_folder = whole_class_directory + "/intensities",
                                          output_regions_folder  = whole_class_directory + "/regionprops",
                                          statistic = 'mean',
                                          re_do = True)

CRC_1_ROI_001.ome intensities csv has been written!
CRC_1_ROI_002.ome intensities csv has been written!
CRC_2_ROI_001.ome intensities csv has been written!
CRC_2_ROI_002.ome intensities csv has been written!
CRC_2_ROI_003.ome intensities csv has been written!
CRC_2_ROI_004.ome intensities csv has been written!
CRC_3_ROI_001.ome intensities csv has been written!
CRC_3_ROI_002.ome intensities csv has been written!
CRC_3_ROI_003.ome intensities csv has been written!
CRC_3_ROI_004.ome intensities csv has been written!
CRC_1_ROI_001.ome regions csv has been written!
CRC_1_ROI_002.ome regions csv has been written!
CRC_2_ROI_001.ome regions csv has been written!
CRC_2_ROI_002.ome regions csv has been written!
CRC_2_ROI_003.ome regions csv has been written!
CRC_2_ROI_004.ome regions csv has been written!
CRC_3_ROI_001.ome regions csv has been written!
CRC_3_ROI_002.ome regions csv has been written!
CRC_3_ROI_003.ome regions csv has been written!
CRC_3_ROI_004.ome regions csv has been written!

class_dict = {1: "lumen", 2: "epithelia", 3: "lamina_propria"}

seed_df = pd.DataFrame()
seed_df['class'] = [i for i in class_dict]
seed_df['labels'] = [class_dict[i] for i in class_dict]
zip_dict = {}
for i,ii in enumerate(seed_df['labels'].unique()):
    zip_dict[ii] = i

seed_df['merging'] = seed_df['labels'].replace(zip_dict)
seed_df


## Each of these dataframes has been created previously -- so in this case I will just retrieve them from the disc
classifier_df = seed_df
metadata = pd.read_csv(project_directory +  "/Analyses/metadata.csv")
panel = pd.read_csv(project_directory +  "/Analyses/Analysis_panel.csv")

metadata.to_csv(whole_class_directory + "/metadata.csv", index = False)
panel.to_csv(whole_class_directory + "/Analysis_panel.csv", index = False)

whole_class_analysis = pbug.WholeClassAnalysis(directory = whole_class_directory, classifier_df = classifier_df, metadata = metadata,
                             Analysis_panel = panel)

Now we can make plots

'''Compare area of each pixel class between conditions:'''
fig = whole_class_analysis.plot_percent_areas()
fig

''' Look at the distribution of markers within a pixel class '''
facet_grid = whole_class_analysis.plot_distribution_exprs(unique_class = 'epithelia', plot_type = 'Violin')
facet_grid.figure

Do Stats

The available statistics are very limited / basic. More advanced statistics should be done with a different software, and given that it is important to check the assumptions of the available statistical tests, it is generally best to do any statistical analysis w9ith at least the help of a differnet software.

whole_class_analysis.whole_marker_exprs_ANOVA(marker_class = 'type', groupby_column = 'class', variable = 'condition', statistic = 'ANOVA', area = True)

	antigen	class	p_adj	p_value	F statistic	avg SSA mean exprs	SSA stdev	avg TA mean exprs	TA stdev
15	CD163	epithelia	2.330000e-08	7.780000e-09	609.000000	0.000481	0.000161	0.00570	0.000494
27	CD31	epithelia	9.460000e-07	3.150000e-07	237.000000	0.001830	0.000828	0.01110	0.001080
85	p-selectin	lamina_propria	1.050000e-06	3.500000e-07	231.000000	0.003850	0.001200	0.01690	0.001530
23	CD206	lumen	1.810000e-06	6.050000e-07	200.000000	0.000653	0.000191	0.00336	0.000417
8	CD11b	lumen	1.870000e-05	6.230000e-06	109.000000	0.001450	0.000872	0.00847	0.001280
...	...	...	...	...	...	...	...	...	...
50	CD8	lumen	9.660000e-01	9.660000e-01	0.001990	0.002090	0.001490	0.00205	0.000876
16	CD163	lamina_propria	9.690000e-01	9.690000e-01	0.001660	0.005430	0.004530	0.00552	0.000517
42	CD66b	epithelia	9.690000e-01	9.690000e-01	0.001590	0.002170	0.002720	0.00223	0.001450
0	Beta-Catenin	epithelia	9.760000e-01	9.760000e-01	0.000944	0.475000	0.113000	0.47200	0.168000
81	areas******	epithelia	9.960000e-01	9.960000e-01	0.000022	26.900000	8.210000	26.80000	6.600000

87 rows × 9 columns

whole_class_analysis.plot_heatmap("p_adj")

Or export the data in tabular form to examine in another software / package:

to_export = whole_class_analysis.export_data(filename = None,           #Or specify a file path to write to, usch as:   whole_class_analysis.directory + "/my_data_export.csv",
                        subset_columns = None,
                        subset_types = None,
                        groupby_columns = None,
                        statistic= 'mean',
                        include_marker_class_row = False)   ## TODO: figure out csv reload with whole class analysis --> for now, not supported
to_export

C:\Users\benca\miniforge3\envs\main\lib\site-packages\anndata\_core\aligned_df.py:68: ImplicitModificationWarning: Transforming to str index.
  warnings.warn("Transforming to str index.", ImplicitModificationWarning)

antigen	aSMA	p-selectin	Vimentin	CD14	CD31	CD16	Pan-Keratin	CD11b	CD163	CD45	...	Seg2	Seg3	index	sample_id	file_name	patient_id	condition	Object	class	areas
0	0.046404	0.002377	0.021375	0.077686	0.001858	0.003592	0.373355	0.003115	0.000352	0.014310	...	0.105902	0.168999	0	0	CRC_1_ROI_001.ome.fcs	7139	SSA	1	lumen	16.497600
1	0.279869	0.004667	0.038276	0.012763	0.003151	0.006478	0.643430	0.007177	0.000741	0.037614	...	0.218496	0.217236	1	0	CRC_1_ROI_001.ome.fcs	7139	SSA	2	epithelia	36.050400
2	0.583634	0.005288	0.231591	0.038185	0.015873	0.008793	0.159639	0.010919	0.002667	0.175283	...	0.200154	0.181208	2	0	CRC_1_ROI_001.ome.fcs	7139	SSA	3	lamina_propria	47.452000
3	0.052761	0.001284	0.016471	0.055562	0.001277	0.001374	0.048797	0.001126	0.000179	0.014350	...	0.067809	0.152127	3	1	CRC_1_ROI_002.ome.fcs	7139	SSA	1	lumen	35.531467
4	0.441734	0.019529	0.028416	0.005586	0.002109	0.003486	0.095743	0.003626	0.000498	0.020471	...	0.148266	0.181678	4	1	CRC_1_ROI_002.ome.fcs	7139	SSA	2	epithelia	15.908267
5	0.461414	0.003406	0.134013	0.023865	0.008549	0.005665	0.036053	0.004593	0.002851	0.090782	...	0.228512	0.158975	5	1	CRC_1_ROI_002.ome.fcs	7139	SSA	3	lamina_propria	48.560267
6	0.043961	0.001348	0.039967	0.017418	0.001145	0.001152	0.063610	0.001428	0.000256	0.030570	...	0.074013	0.166255	6	2	CRC_2_ROI_001.ome.fcs	7139	SSA	1	lumen	40.148800
7	0.050636	0.002190	0.015143	0.004244	0.001938	0.002836	0.239402	0.003481	0.000497	0.020262	...	0.093077	0.189527	7	2	CRC_2_ROI_001.ome.fcs	7139	SSA	2	epithelia	36.967600
8	0.385122	0.003522	0.240166	0.021005	0.011558	0.004695	0.016083	0.007222	0.003367	0.160490	...	0.135513	0.174451	8	2	CRC_2_ROI_001.ome.fcs	7139	SSA	3	lamina_propria	22.883600
9	0.048720	0.001244	0.017306	0.027874	0.000866	0.001455	0.084637	0.000899	0.000146	0.005761	...	0.128837	0.157224	9	3	CRC_2_ROI_002.ome.fcs	7139	SSA	1	lumen	44.958000
10	0.238822	0.002845	0.017037	0.005089	0.001525	0.003130	0.157704	0.002277	0.000392	0.008053	...	0.175399	0.180829	10	3	CRC_2_ROI_002.ome.fcs	7139	SSA	2	epithelia	22.159600
11	0.531417	0.005083	0.291689	0.020345	0.005762	0.005054	0.044761	0.004031	0.001856	0.093584	...	0.253888	0.167786	11	3	CRC_2_ROI_002.ome.fcs	7139	SSA	3	lamina_propria	32.882400
12	0.035091	0.001602	0.027311	0.082346	0.001521	0.001601	0.082200	0.001488	0.000326	0.026056	...	0.047149	0.152558	12	4	CRC_2_ROI_003.ome.fcs	7139	SSA	1	lumen	7.479200
13	0.110517	0.002089	0.033115	0.007243	0.001643	0.003004	0.138882	0.004171	0.000505	0.026921	...	0.157469	0.205587	13	4	CRC_2_ROI_003.ome.fcs	7139	SSA	2	epithelia	25.846400
14	0.338910	0.003757	0.216403	0.024321	0.008768	0.011529	0.010334	0.009493	0.008544	0.156488	...	0.262179	0.173224	14	4	CRC_2_ROI_003.ome.fcs	7139	SSA	3	lamina_propria	66.674400
15	0.031635	0.006760	0.021256	1.680096	0.053469	0.009906	0.005348	0.000664	0.001362	0.003080	...	0.016429	0.148124	15	5	CRC_2_ROI_004.ome.fcs	7139	SSA	1	lumen	4.778909
16	0.183516	0.001641	0.016439	0.003520	0.000621	0.000745	0.025288	0.000837	0.000252	0.002806	...	0.047211	0.177667	16	5	CRC_2_ROI_004.ome.fcs	7139	SSA	2	epithelia	24.179636
17	0.217762	0.002035	0.193029	0.015542	0.006711	0.002814	0.003055	0.004390	0.013288	0.042991	...	0.038650	0.154528	17	5	CRC_2_ROI_004.ome.fcs	7139	SSA	3	lamina_propria	71.041455
18	0.062293	0.016413	0.100151	0.042670	0.007151	0.009943	0.107857	0.009370	0.003144	0.177840	...	0.084772	0.163744	18	6	CRC_3_ROI_001.ome.fcs	7139	TA	1	lumen	15.121000
19	0.191357	0.019775	0.057836	0.018934	0.012637	0.015682	0.242961	0.015912	0.005724	0.105703	...	0.103431	0.200553	19	6	CRC_3_ROI_001.ome.fcs	7139	TA	2	epithelia	34.485600
20	0.281676	0.018888	0.201009	0.027490	0.015082	0.020664	0.100855	0.017962	0.005775	0.407615	...	0.183162	0.180404	20	6	CRC_3_ROI_001.ome.fcs	7139	TA	3	lamina_propria	50.393400
21	0.061697	0.015720	0.084369	0.089420	0.006197	0.009044	0.124590	0.009496	0.003162	0.076874	...	0.106464	0.143011	21	7	CRC_3_ROI_002.ome.fcs	7139	TA	1	lumen	12.536600
22	0.167929	0.018500	0.051282	0.016637	0.010217	0.013284	0.239551	0.012887	0.005303	0.057870	...	0.092356	0.164743	22	7	CRC_3_ROI_002.ome.fcs	7139	TA	2	epithelia	29.006200
23	0.208419	0.015966	0.149385	0.024263	0.013647	0.018055	0.104513	0.016503	0.005405	0.226643	...	0.169895	0.153509	23	7	CRC_3_ROI_002.ome.fcs	7139	TA	3	lamina_propria	58.457200
24	0.053290	0.013046	0.022435	0.080215	0.005233	0.007236	0.041152	0.006740	0.002285	0.012031	...	0.045199	0.137282	24	8	CRC_3_ROI_003.ome.fcs	7139	TA	1	lumen	16.764000
25	0.207609	0.015706	0.025147	0.013484	0.010499	0.012301	0.088116	0.010148	0.005396	0.020136	...	0.088967	0.151050	25	8	CRC_3_ROI_003.ome.fcs	7139	TA	2	epithelia	24.993600
26	0.318935	0.017284	0.279070	0.029856	0.020118	0.016477	0.039863	0.013732	0.006054	0.100555	...	0.160039	0.145882	26	8	CRC_3_ROI_003.ome.fcs	7139	TA	3	lamina_propria	58.242400
27	0.050464	0.015304	0.030541	0.016861	0.005928	0.008065	0.062523	0.008266	0.003270	0.038459	...	0.043146	0.134916	27	9	CRC_3_ROI_004.ome.fcs	7139	TA	1	lumen	26.593600
28	0.114999	0.019065	0.041749	0.016181	0.010985	0.012104	0.159120	0.011640	0.006394	0.057640	...	0.052795	0.141570	28	9	CRC_3_ROI_004.ome.fcs	7139	TA	2	epithelia	18.830600
29	0.230477	0.015477	0.069437	0.018115	0.013465	0.013979	0.053264	0.014817	0.004859	0.148455	...	0.101144	0.142696	29	9	CRC_3_ROI_004.ome.fcs	7139	TA	3	lamina_propria	54.575800

30 rows × 42 columns