==== Front BMC Med Inform Decis Mak BMC Med Inform Decis Mak BMC Medical Informatics and Decision Making 1472-6947 BioMed Central London 1327 10.1186/s12911-020-01327-3 Research Texture features in the Shearlet domain for histopathological image classification Alinsaif Sadiq salin025@uottawa.ca Lang Jochen jlang@uottawa.ca grid.28046.380000 0001 2182 2255EECS, University of Ottawa, Ottawa, Canada 15 12 2020 15 12 2020 2020 20 Suppl 14 Publication of this supplement has not been supported by sponsorship. Information about the source of funding for publication charges can be found in the individual articles. The articles have undergone the journal's standard peer review process for supplements. The Supplement Editor declares no competing interests.312© The Author(s) 2020Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.Background A various number of imaging modalities are available (e.g., magnetic resonance, x-ray, ultrasound, and biopsy) where each modality can reveal different structural aspects of tissues. However, the analysis of histological slide images that are captured using a biopsy is considered the gold standard to determine whether cancer exists. Furthermore, it can reveal the stage of cancer. Therefore, supervised machine learning can be used to classify histopathological tissues. Several computational techniques have been proposed to study histopathological images with varying levels of success. Often handcrafted techniques based on texture analysis are proposed to classify histopathological tissues which can be used with supervised machine learning. Methods In this paper, we construct a novel feature space to automate the classification of tissues in histology images. Our feature representation is to integrate various features sets into a new texture feature representation. All of our descriptors are computed in the complex Shearlet domain. With complex coefficients, we investigate not only the use of magnitude coefficients, but also study the effectiveness of incorporating the relative phase (RP) coefficients to create the input feature vector. In our study, four texture-based descriptors are extracted from the Shearlet coefficients: co-occurrence texture features, Local Binary Patterns, Local Oriented Statistic Information Booster, and segmentation-based Fractal Texture Analysis. Each set of these attributes captures significant local and global statistics. Therefore, we study them individually, but additionally integrate them to boost the accuracy of classifying the histopathology tissues while being fed to classical classifiers. To tackle the problem of high-dimensionality, our proposed feature space is reduced using principal component analysis. In our study, we use two classifiers to indicate the success of our proposed feature representation: Support Vector Machine (SVM) and Decision Tree Bagger (DTB). Results Our feature representation delivered high performance when used on four public datasets. As such, the best achieved accuracy: multi-class Kather (i.e., 92.56%), BreakHis (i.e., 91.73%), Epistroma (i.e., 98.04%), Warwick-QU (i.e., 96.29%). Conclusions Our proposed method in the Shearlet domain for the classification of histopathological images proved to be effective when it was investigated on four different datasets that exhibit different levels of complexity. Keywords Texture descriptorsHistologyComplex shearletClassificationSVM10th International Workshop on Biomedical and Health Informatics San Diego, CA, USA 18-20 November 2019 issue-copyright-statement© The Author(s) 2020 ==== Body Background In medical imaging, the study of histology images is considered a significant task [1]. The advancement of technology allows the histological slides to be digitized and stored in digital form [2]. The inspection of histological slides manually by a histopathologist is indispensable. However, computational techniques from image processing and machine learning can be of a great asset in the field of histopathology to assist in applying pre-screening/classification of easy cases. Therefore, more time can be consumed in studying the challenging histological slides. More importantly, computer-assisted diagnosis in histopathology can play a significant role in minimizing (and ultimately eradicating) man-made mistakes, e.g. by the pathologist [3]. Therefore, the early identification of cancer is crucial for the pathologist to propose an appropriate treatment for the patients. The process of histopathological tissue classification is tackled in different ways. We divide our review of such techniques into three groups: texture-based, Shearlet-based, and deep feature-based methods. Texture-based techniques are frequently investigated for the analysis and classification of histopathological tissues. For instance, Kather et al. [4] proposed computing various texture features and classify colorectal cancer histology using SVM (i.e., using 10-fold cross-validation) into eight classes. The fusion of different texture features delivered an accuracy of 87.4%. Similarly, Linder et al. [5] investigated a different number of descriptors: LBP, Haralick texture attributes, and Gabor filters to classify their introduced dataset which is called, Epistroma. As such, those extracted descriptors are fed to an SVM model to distinguish between epithelium and stroma tissues. Comparably, Spanhol et al. [3] established a new dataset called, breast cancer histopathology dataset (BreakHis). This dataset consists of benign and malignant tissues. Spanhol et al. used different techniques to classify BreakHis tissues into benign or malignant. Bruno et al. [6] proposed applying LBP on the curvelet coefficients of the transformed image)(i.e., authors used the Digital Database for Screening Mammography, Breast Cancer Digital Repository, and UCSB biosegmentation benchmark). To reduce the number of descriptors, the authors used statistical analysis of variance (ANOVA). In comparison to Bruno et al., we also use descriptors computed from a directional wavelet transform, but we demonstrate that it is advantageous to integrate various descriptors computed in the Shearlet domain to create the feature space. A more relative idea to our technique is proposed by Ribeiro et al. [7]. Ribeiro et al. computed descriptors from both spatial images and curvelet coefficients to classify colorectal histology tissues. In contrast to Ribeiro et al., we utilized both magnitude and phase coefficients of the complex Shearlet domain. A similar approach to ours is proposed by Vo et al. [8] who extracted both, phase and magnitude descriptors for textured image retrieval, but applied to non-medical texture image samples. The Shearlet transform has been previously used in different studies. Such a transform has the advantage of constructing an anisotropic system of a wavelet. However, only the magnitude coefficients are utilized for the classification of textured images. For instance, He et al. [9] classified textured images by proposing Shearlet-based descriptors. Authors in this study, quantize and encode the local energy descriptors computed from the Shearlet coefficients. Thereafter, the energy histograms of all levels are cumulated to form the image characteristics. Instead, Zhou et al. [10] utilized only specific levels of the decomposition of the Shearlet domain for breast tumor ultrasound image classification. Dong et al. [11] suggested a technique for textured images classification and retrieval, where the dependencies of adjacent Shearlet subbands are modeled using linear regression. To represent the Shearlet subbands for classification, energy descriptors are computed. However, the textured image retrieval consists of both statistics in the contourlet and Shearlet domains. Meshkini and Ghassemian [12] proposed to classify textured images using the inner product of the co-occurrence matrix and magnitude coefficients of Shearlet transform. Different to many published studies, we do not only use the magnitude coefficients, but also the phase coefficients of the Shearlet transform in our work. Deep feature descriptors which are extracted from a pre-trained deep learning model (particularly, convolutional neural network (CNN)) which are typically trained on non-medical images. As such, these models are either used without fine-tuning (i.e., as unsupervised feature extractors) or fully/partially retrained for biomedical images. For example, Song et al. [13] proposed classifying the BreakHis dataset using feature vectors extracted from a CNN. As such, the extracted descriptors from the CNN are encoded using the Fisher Vector method. Similarly, Gupta et al. [14] extracted deep features from a fine-tuned DenseNet, but from multiple layers to classify BreakHis dataset. Differently, Wang et al. [15] utilized color deconvolution to obtain the hematoxylin and eosin channels separately. Subsequently, one CNN model is trained using hematoxylin components and another CNN trained using eosin components, and finally, the outputs of the two CNNs are fused to obtain the final prediction.Fig. 1 Summary of our work approach As has been discussed above, computational techniques have been applied previously to predict the class of a tissue type in histological images. As such, conventional and deep learning (DL) techniques have been developed [16]. However, with the scarcity of well-curated histopathological datasets for training/testing deep neural networks  [17, 18], training a deep learning model can be a challenging approach. Therefore, in our study, we present a novel Shearlet-based hand-engineered texture descriptors to classify tissue types. Namely, co-occurrence texture descriptors [19, 20], Local Binary Patterns (LBP) [21], Local Oriented Statistics Information Booster (LOSIB) [22], and Segmentation-based Fractal Texture Analysis (SFTA) [23] are used in our study which each of these set of descriptors are computed in the Shearlet domain [24]. Then, these features are utilized to train/test two classifiers: Support Vector Machine (SVM) [25] and a Decision Tree Bagger (DTB) [26]. Notably, our modeling technique is taking advantage of the directionality in the complex Shearlet transform where we utilized both the magnitude and phase coefficients. As such, those coefficients are summarized using various textural methods that can capture local and global attributes [19, 23] of histopathological tissues. Most interestingly, computing such statistics from the directional sub-bands can potentially lead to capturing significant information that can be missed in the spatial domain because of the complexity of histopathological tissues. In our research, we investigate both parametric and non-parametric (i.e., robust in classification [27]) classification models. We include DTB because it is a ML method that is considered to lead to explainable decisions, unlike SVM which is considered as a block box classifier [28]. We show that the fusion of some sets of descriptors can result in a vigorous feature representation. Thereafter, we employ principal component analysis to further enhance the classification results while having a reduced set of features.Fig. 2 Samples of each dataset that we have used in our study This paper is an extension of work previously presented at the Biomedical and Health Informatics (BHI) Workshop 2019 [29]. Our main contributions in this extension are summarized below:We propose and present a comprehensive justification for our feature space, namely, the Shearlet-based texture descriptors for histopathological image classification. We demonstrate that when these attributes are used to train a conventional ML model (i.e., SVM and DTB in this extended version), they provide better classification performance than several existing methods on the four standard datasets used in this research. We present an extended study of our feature representations expressed in principal components that decreases computational cost without significantly compromising accuracy. Methods Figure 1 provides an overview of our proposed system for the classification of histopathological images. A detailed description of each component of our method is provided in the following sections. MATLAB® 2017b is utilized for the implementation of our techniques. Our proposed method consists of three steps:Step 1: For a given histopathological image, we apply the complex Shearlet transform. With the complex coefficients, we calculate the magnitude and relative phase (RP). Step 2: We then extract four sets of features from the directional sub-bands of the RP and magnitude: co-occurrence based texture features, local binary pattern, local oriented statistics information, and segmentation-based fractal texture analysis. Step 3: We then apply one of two different classifiers: DTB or SVM. Datasets Our proposed technique has been evaluated on four different histopathological datasets that exhibit different levels of complexity: Multi-class Kather’s dataset [4] consists of a total of 5000 images (i.e., each image has a size of 150×150 pixels). This dataset provides tissue types that belong to 8-types (i.e., tumor epithelium, simple stroma, complex stroma, immune cells, debris, normal mucosal glands, adipose tissue, and background (no tissue)) - there are 625 tissue samples for each type. An example of each tissue type is shown in Fig. 2a. Breast Cancer Histopathological dataset (BreakHis) dataset [3] contains tissue from two categories: benign and malignant breast tumors (Examples are shown in Fig. 2b). The total number of samples from two tissue types is 7909 images (i.e., each image has a size of 700×460 pixels). It worth noting that each histological slide is captured and stored with various magnification factors: 40×, 100×, 200×, and 400×. The distribution of samples of the benign/malignant in each magnification factor as follows: 625/1370(40×), 644/1437(100×), 623/1390(200×) and 588/1232(400×). Spanhol et al. [3] proposed to use each magnification factor as a separate dataset. However, in our study, we combine all magnification factors as one dataset as Jonnalagedda [30]. This is motivated by the fact that each magnification factor captures different information [31]. Epistroma dataset contains variable size histo-pathology images that belong to two tissue types (as shown in Fig. 2c): stroma (551 samples) and epithelium (825 samples) [5]. Warwick-QU dataset obtained with a magnification factor of 20× from colon histology sections. It is a binary dataset of benign (74 samples) or malignant (91 samples) [32]. Examples of the tissue types are shown in Fig. 2d. Complex Shearlet transform In this study, we present a new perspective for computing attributes that summarize the statistical information/distribution of the Shearlet magnitude/phase coefficients for every scale and orientation [24]. Given a complex coefficient, C=x+iy where first term express the real part and the second term express the imaginary part, then we can compute the magnitude (as ρ=x2+y22) and phase (as θ=tan-1(y/x)) components. In contrast to existing studies [9, 10], we do not only use the energy of the complex Shearlet transform, but we also examine the usefulness of the phase components and their potential for providing more robust characterization for medical image classification. We experimentally verify that using phase alongside with magnitude coefficients can, in fact, boost the classification performance (See Section ). Our work is motivated by research completed by Vo et al. [8]. Vo et al. build a feature space (i.e., consist of magnitude and relative phase (RP)) that is computed from a complex directional filter bank for textured image retrieval. However, In our study, we acquire such an idea but alternatively compute the relative Shearlet phase components. The complex Shearlet transform can be applied on a histopathological image that has a size of M×M to be transformed to S scales where every scale consists of K directionalities. Let θsk(i,j) at location (i, j) to represent the phase angle component at at scale s and directionality k, where s=1,2,...,S and k=1,2,...,K. In our study, we use S=4 and K=8 per scale. Now, we can compute the relative Shearlet phase for a phase component at position (i, j) of a directional sub-band in the following manner: 1 RPsk(i,j)=θsk(i,j)-θsk(i,j+1),if1⩽k⩽K2.θsk(i,j)-θsk(i+1,j),ifK2