
==== Front
Heliyon
Heliyon
Heliyon
2405-8440
Elsevier

S2405-8440(24)12923-4
10.1016/j.heliyon.2024.e36892
e36892
Research Article
ArSa-Tweets: A novel Arabic sarcasm detection system based on deep learning model
Abuein Qusai qabuein@just.edu.jo
a
Al-Khatib Ra'ed M. raed.m.alkhatib@yu.edu.jo
b⁎
Migdady Aya ayamigdadiha@gmail.com
a
Jawarneh Mahmoud S. m_jawarneh@asu.edu.jo
c
Al-Khateeb Asef asef.alkhateeb@jpu.edu.jo
d
a Department of Computer Information Systems, Jordan University of Science and Technology, Irbid, 22110, Jordan
b Department of Computer Sciences, Yarmouk University, Irbid, 21163, Jordan
c Faculty of Information Technology, Applied Science Private University, Amman, Jordan
d Faculty of Computer Science and Information Technology, Jerash University, Jerash, Jordan
⁎ Corresponding author. raed.m.alkhatib@yu.edu.jo
28 8 2024
15 9 2024
28 8 2024
10 17 e3689215 3 2024
17 8 2024
23 8 2024
© 2024 The Author(s)
2024
https://creativecommons.org/licenses/by-nc/4.0/ This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).
Sarcasm in Sentiment Analysis (SA) is important due to the sense of sarcasm in sentences that differs from their literal meaning. Analysis of Arabic sarcasm still has many challenges like implicit indirect idioms to express the opinion, and lack of Arabic sarcasm corpus. In this paper, we proposed a new detecting model for sarcasm in Arabic tweets called the ArSa-Tweet model. It is based on implementing and developing Deep Learning (DL) models to classify tweets as sarcastic or not. The development of our proposed model consists of adding main improvements by applying robust preprocessing steps before feeding the data to the adapted DL models. The adapted DL models are LSTM, Multi-headed CNN-LSTM-GRU, BERT, AraBert-V01, and AraBert-V02. In addition, we proposed ArSa-data as a golden corpus that consists of Arabic tweets. A comparative process shows that our proposed ArSa-Tweet method has the most impact accuracy rate based on deploying the AraBert-V02 model, which obtains the best performance results in all accuracy metrics when compared with other methods.

Keywords

Deep learning (DL)
Sarcasm
Sentiment analysis (SA)
Machine learning
Natural language processing (NLP)
Tweets
==== Body
pmc1 Introduction

Sarcasm is an important field in the Sentiment Analysis (SA) domain, where the sense of sarcasm in sentences differs from their literal meaning. Analysis of Arabic sarcasm faces many challenges. The implicit indirect idioms to express the opinion made the sarcasm analyses more sophisticated. The world of linguistics in literature has recently witnessed rapid and tremendous development in language analysis such as SA. Based on [1], SA is a method in which emotion is extracted and analyzed from a specific part of the text. One of the hot themes in SA is sarcasm analysis which gained great resonance in recent research. The reason behind its importance is the tremendous use of several platforms for public social media. Meanwhile, scripts issued on public platforms like Facebook and Twitter contain expressive and metaphorical language like sarcasm and irony. While SA interprets and classifies sentences as positive, negative, or neutral emotions, the text could be said as ironic if the sentence's meaning is the reverse of what is written literally. So, techniques of irony detection in extracting information from scripts increase its importance and value. For instance, to transcend the exact match of user inquiries, irony recognition will improve information recovery among recent operators to promote the non-exact retrieval of inventive terms.

The work in [2] defined sarcasm as a verbal irony formula that refers to disdain or mockery. On the other hand, the research in [3] defined sarcasm as the way of expressing a view/thinking indirectly, but the purposes of the concept are opposite. Moreover, sarcasm usually depends on the context, because it is always a part between the parties and the common knowledge [2]. Also, sarcasm recognition is a critical mission in the SA domain. Although the satirical utterances are expressed in a positive expression, they hold with them an implicit negative feeling. This inconsistency between the exterior sentiment with the implicit meaning produces a sophisticated issue for SA procedures [4].

Moreover, several studies adopted English sarcasm recognition tasks, either in collecting data levels, or sarcasm identification models. On the other hand, Arabic sarcasm works are still under development. As such, the number of studies that adopted sarcasm sentiment analysis is very scarce, and to the best of our knowledge can be counted on a few of works such as [5], [6], [7] (discussed on Section 2). Further, the publicly available corpora for the Arabic sarcasm system are still limited in these researches [8], [7], (discussed in Section 2).

Our work in this paper is to propose a newly developed Arabic sarcasm detection method called ArSa-Tweets model. This proposed ArSa-Tweets model is also harnessed with a new dataset as a golden corpus called “ArSa-data”, which is collected using the Twitter Application Programming Interface (API). The services of Twitter API will be discussed in detail in section 3.2.1. The collected corpus, ‘ArSa-data’, has been manually annotated by Arabic experts. ArSa-data contains 3,000 tweets, 1,004 (33.5%) of which are sarcastic. The primary goal of our research paper is to discover the sarcastic sentiments of individuals everywhere, For that reasons, we use sarcastic Arabic keywords such as “Image 1 Madraseh” and “Image 2 Reyadah”, which mean in English “school” and “sport”, respectively.

The following points summarize the main contributions to fulfill the basic goal of this study:1. Proposing new preprocessing architecture to enhance the quality of extracted data by Principal Component Analysis (PCA) features reduction technique.

2. Deploying the most efficient DL models including Multi-headed CNN-LSTM-GRU, BERT, LSTM, AraBert-V01, and AraBert-V02, also, examining our proposed preprocessing system based on deploying these five DL models.

3. Using three other publicly available sarcasm datasets, as well as using the main accuracy metrics for benchmark comparison against some related works done on such shared datasets.

The statistical analysis and the algorithms used proved that the sarcastic views of people in both fields (school and sport) have a significant impact on society, and it is worthy to be studied. Moreover, our proposed development methodology obtained the best results based on deploying the LSTM, Multi-headed CNN-LSTM-GRU, BERT, AraBert-V01, and AraBert-V02 as DL algorithms.

The rest of this work is categorized into several sections. The literature works related to Arabic sarcasm recognition are introduced in Section 2. A full description of the proposed ArSa-Tweets development based on DL models is discussed in Section 3. The performance outcomes are shown and argued in Section 4 using the collected ArSa-data and other available datasets like ArSarcasm-v1, ArSarcasm-v2, and SemEval-2022. The final section introduces the conclusion which recaps the places of interest of such study and the plan for some future research directions.

2 Related works

The English-published research has a lot of works on irony and sarcasm recognition, that differ from gathering the proper corpus to construction identification systems. For example, the research in [9] is considered as one of the initial English sarcasm/irony detection works, in which the authors produced a Twitter corpus by using particular hashtags like (#sarcasm and #not), this point out the irony issue [10]. However, these kinds of data-gathering approaches are based on particular content that appears on it, which is called distant supervision approaches. This kind of distant supervision is one of the most popular ways to gather irony corpus from the Twitter platform, in which the keyword hashtag (#sarcasm and others) are commonly used.

The process of hashtags is useful in a situation when the hidden sarcasm is not clear. The work in [5] adopted the supervision approach to collect data. Also, the authors developed another corpus collected from the Reddit site, which is based on a remote supervision approach. Unless the proposed approach depends on “/s” symbols as irony indicators. Moreover, manual labeling is considered another way of creating the dataset, which can be done by gathering the corpus and asking annotators to label it manually. This kind of labeling approach is primarily based on the ability of annotators to capture the sarcastic sentence whether it is explicit or intended. On the other side, some works have been recently presented in the Arabic sarcasm detection system. The following subsequently describes some of them.

The work in [11] adapted DL-based models for detecting Arabic tweets. Their system was issued by the RGCL team via Irony Detection in Arabic Tweets (IDAT 2019). They tested how the text preprocessing and the text cleaning could enhance the accuracy of their system. They got a good F1_score that was obtained among several runs submitted was 81.8%.

In [12], the authors presented three different types of ensemble models as an ensemble learning technique to perform sarcasm detection tasks. These three models are named classical, deep, and hybrid. The results of three runs were submitted to recognize the Arabic Tweets at the FIRE 2019 Forum in the Information Retrieval (IR) domain. They extracted from the tweets, various types of features containing topic modeling features, word n-gram features based on the TF-IDF model, bag-of-words, and sentiment features [13], [14]. They achieved 84.4% F1_score on the experimental results.

In [7], the researchers introduced a model called ArSarcasm based on creating an Arabic sarcasm dataset. This model was produced by the re-labeling of the Arabic sentiment analysis datasets. It includes annotated data that has 10,547, where 16% of it is sarcastic. Not only the data was annotated for sarcasm, but it was also labeled for sentiment and dialects. Their approach presented an extremely internal nature in such work, where it clears from the change in sentiment annotations founded on annotators' biases. This trial displayed the decline of state-of-the-art sentiment analysts against sarcastic content. Then, they used a BiLSTM deep learning network for irony recognition, where their approach obtained an F1_score of 46%.

In [6], the authors presented a method to enhance Arabic sarcasm detection. Their work is based on using the Random Forests model with data augmentation and contextual word embedding to attain the best results. They used the shared dataset ArSarcasmV2 to assess their work and they achieved 75.33% in accuracy, and the F1_score reached 51.89%.

The research in [15] developed a model based on an analysis of tweets regarding the newly emerging Coronavirus analysts (COVID-19) pandemic. They designed an automatically model to identify and understand the user goal of tweets, which could be a harmful level they can inflect. So, they constructed and released AraCOVID19-SSD1, which is a manually labeled Arabic COVID-19 irony, and then they labeled the sentiment corpus. Their dataset has 5,162 records of tweets. Support Vector Machine (SVM), Random Forests model (RF), and Logistic Regression (LR) are used as classification systems for testing and analyzing the used dataset [16]. Also, in [17] Arabic classifiers were tested using six different supervised machine learning classification algorithms: Random Forest (RF), K-Nearest Neighbor (KNN), Decision Tree (DT), Support Vector Machine (SVM), Multinomial Naïve Bayes (NB), Logistic Regression (LR), and TF-IDF are used for feature selection. Additionally, they used different N-grams for tokenizing.

In [18] presented a novel method for the semi-supervised learning of sarcastic Arabic hotel evaluations, which are based on domain adaption and use the self-training strategy [19]. For contextual embedding, they employed AraBERT, and for self-training categorization, they used machine learning models.

Three pre-trained Arabic transformer-based Language Models (LM) for Arabic sarcasm detection were improved, and a new Arabic sarcastic corpus was created in [20]. Also, they presented a hybrid deep learning method for sarcasm detection. Many pre-trained language models (LM) are deployed such as ‘Word2Vec’ word embeddings, and Bidirectional Encoder Representations from Transformers (BERT) models for pre-trained on Arabic resources to mix static and contextualized representations.

The study of how removing emojis from datasets affects how well sarcasm recognition algorithms perform in social media, can be found in [21]. To increase sarcasm detection skills, this work involves modifying and improving AraBERT pre-training models, particularly by removing emojis. Therefore, they employed AraBERT pre-training model to fine-tune the given models, showing that emoji removal can dramatically improve sarcasm recognition accuracy.

However, Arabic sarcasm recognition is still in its infancy. The fact that there are not many publicly accessible corpora highlights the difficulty of finding Arabic literature on sarcasm [22]. Furthermore, in addition to language structure and morphology–which are more complicated in Arabic–sarcasm issues are closely linked to context and culture. Performance is dependent on several variables. Examples include the size of the corpus, the parameters of the model, preprocessing, and methods for extracting features, in addition to the machine resources. This suggests that every experiment is independent of the others and that we cannot claim that one model performs better than the others until we assume that every variable stays the same.

To this end, we propose a new detection method for Arabic sarcasm based on deep-learning models. We explain our proposed enhancement DL-based model by incorporating the LSTM, Multi-headed CNN-LSTM-GRU, BERT, AraBert-V01, and AraBert-V02. Also, we collect a new Arabic sarcasm dataset, called ArSA-data as a new golden corpus, based on two contexts and different cultures. Further, we use an algorithmic features extracted method that relied on the PCA approach, in addition to proposing a novel developed preprocessing model for manipulating the collected dataset. So, the procedure of collecting and classifying the tweets in our proposed system approach is shown in Fig. 1.Figure 1 Procedure of collecting and classifying the tweets in the proposed system. The proposed procedure starts by collecting new data from Twitter platforms by using its available public API. Then, Python programming languages are used for query implementations. The retrieved text data of tweets with the ‘hashtag’, ‘id’, ‘location’, and the ‘posted date of tweets’ are saved on an Excel file. Next, the data is cleaned and some preprocessing steps have been performed in addition to dividing the data into training, validation, and testing sets. Finally, we evaluate our proposed ArSa-Tweets detection system by training the new data over five Deep Learning models (LSTM, Multi-headed CNN-LSTM-GRU, BERT, AraBert-V01, and AraBert-V02), and then we report the outcomes based on four metrics (Accuracy, Precision, Recall, and F1-score).

Figure 1

Firstly, our proposed sarcasm detection systems are described, and then we explain the process followed to extract the data in addition to explaining how the data is labeled. Finally, the processing applied over the corpus is also described in this section.

3 Methodology

3.1 Simulation design

The ArSa-data dataset in addition to the three publicly available datasets (ArSarcasm-v1, ArSarcasm-v2, and SemEval-2022) was used to simulate the suggested models using the Python computer language. The research utilized a Windows 10 64-bit system platform for software applications, and an NVIDIA GeForce MX130 with 2 GB VRAM, 32 GB RAM with an Intel(R) Core(TM) i5=8250U CPU that runs at 1.60 GHz and 1.80 GHz for hardware [23]. Python version 3.8 and the Keras framework are used to implement experiments in the Jupyter Notebook environment.

3.2 Dataset

3.2.1 Collection with annotation

In this research, we collect new data from one of the major social platforms that allows posting both text and hashtags, called Twitter (TW) by using its available public API. This data is collected as a golden corpus called ArSa-data, in order to test our proposed model and to be used by other researchers. Basically, millions of users post their tweets every day which produces a huge number of tweets shared online. These tweets contain a wide variety of topics such as technology, movies, sports, politics, poetry, etc. One of the most interesting features for users but poses a challenge for developers, is the ability to write tweets in any language and any dialect on the Twitter platform. Another interesting feature is the ability of programmers to use two available public APIs, which are Search API and Streaming API. Both of them are used for different purposes. To extract historical data, Search API is used. On the other hand, Streaming API is used to acquire live data. To retrieve the required information from specific users or from tweets that contain a certain keyword or hashtag, various filters can be applied to get this information. Moreover, it is possible to retrieve tweets for a particular date or date range by using Search API.

For authentication purposes, Twitter provides developers with an API key and API secret to use Search API. The first step is to get the API key. While the secret API is important in the creation of a Twitter developer account. Then by utilizing the key of API, the query sentence can be implemented and used in different programming languages. In this study, Python programming languages are used for query implementations. We used two non-sarcastic keywords: “Image 1 Madrasah” and “Image 2 Reyadah” in the query, which sentenced to retrieve various tweets topics. The process of data retrieval has a text of tweets with the ‘hashtag’, ‘id’, ‘location’, and the ‘posted date of tweets’. The tweet's texts were manually annotated as sarcastic or not by two Arabic language specialists. In our proposed model, the data is cleaned, and some preprocessing steps have been performed such as: removing duplicate tweets, cleaning text tweets, and extracting the emoji from the texts. The main steps to gather and generate the dataset are demonstrated in Fig. 1. Consequently, we collect and present this dataset to make it appropriate for sarcasm analysis. Therefore, we split the tweets into text and emoji into two separate columns. These emojis will aid in classifying tweets in sarcasm or not classing depending on the sarcastic score of the emoji. Algorithm 1 has the main steps of manipulating data in the scraping stage.Algorithm 1 Preprocessing for collecting tweets data in our proposed ArSa-Tweets system.

Algorithm 1

3.2.2 Statistics and analysis

The final processed dataset contains 3,000 records (tweets). Each of them has been labeled as sarcastic or not. Table 1 illustrates the statistical analysis of the ArSa-data tweets. Clearly, from Table 1, we find that 33.5% of the tweets are irony (1,004 tweets). Fig. 2 shows a pie-chart for the manipulation process of our proposed ArSa-data that are reported in Table 1, as seen most of the tweets are not sarcastic.Table 1 Sarcasm distribution over ArSa-Tweets.

Table 1Tweets	# of Tweets	Percentage	
Sarcastic	1,004	33.5%	
Not-sarcastic	1,996	66.5%	
Total	3,000	100%	

Figure 2 Pie chart of sarcasm distribution over our proposed ArSa-data corpus of tweets. The dataset contains 3,000 tweets, 1,004 (33.5%) are sarcastic tweets and 1996 (66.5%) are not sarcastic tweets.

Figure 2

Table 2 displays some samples of sarcastic tweets from different dialects including their emojis. They provide the main characteristics and features of a sarcastic nature. Working on classifying tweets as sarcastic or not is an arduous process, which depends on the nature of the written speech, while the written may appear as normal speech until it is filled with a kind of irony. Actually, it could be recognized as metaphoric or figurative terminologies, which could fall under the irony definition. This poses a great challenge for the researcher who is looking for a model capable to address such a problem and efficiently classify tweets.Table 2 Samples of sarcastic example from ArSa-Tweets dataset.

Table 2

This case is proved in text five at Table 2, where the person (who raises the tweet) uses seemingly promising and positive words to advise others about the importance of sports, but it is a line of irony. However, all of the examples in Table 2 display how sophisticated irony tweet detection can be. Where some tweets are written using serious expressions, even though gold sarcastic sentiments, and vice versa. As such, it particularly adds other challenges, which increase the necessity for building more efficient irony recognition systems. It shows that irony depends extremely on common knowledge and text context. As such, combining this information is considered crucial to efficiently recognizing irony.

3.3 Parameter assignments

LSTM model: We decided to use an embedding with dimension 300 followed by a single LSTM layer, after experimenting with several architectures. This allowed us to reduce training time and observe how word embedding affected the models' performance. The result of the sarcastic categorization is given by the dense output.

Convolutional Neural Networks (CNN): The model consists of a pooling operation, a non-linearity layer (often ReLU), and a convolutional layer applied to a set of pre-trained word embeddings.

The following training parameters are applied to the LSTM and Multi-headed-LSTM-CNN-GRU neural models:1. Loss function: the Binary Cross Entropy was chosen as the loss function because we are training a binary classifier. This allows us to reduce error and modify model weights as needed throughout training.

2. Optimization algorithm: The Adam method is the method that we employed. SGD is also a simple, infrequently applied algorithm. Adam and RMSProp have produced quality work. However, Adam is our choice since it is frequently employed for comparable jobs and consistently produces quick, excellent outcomes.

BERT Model: is offered by the Hugging Face Transformer library, which has been trained on larger unlabeled text corpora, such as Wikipedia and book corpora [13], [14], [15]. The weights of pre-trained BERT model are initialized in order to carry out the fine-tuning. Using labeled data, all 109 million parameters are adjusted precisely. In order to generate an Arabic BERT that is specifically utilized for Arabic tweet classification, a dense classification layer is added to the encoder. The 12 encoder layers of the customized BERT model have 12 bidirectional self-attention heads and 12 hidden layers. The encoder structure passes through the final embedding layers that are created after combining with segment and positional embedding. Lastly, Arabic tweets are categorized using the Softmax classification layer.

AraBERT Model: adding a fully connected layer to the pre-trained model is deployed for fine-tuning. Each model is then improved for a particular purpose. We use the HuggingFace package, which offers a straightforward, high-level TensorFlow interface for importing a variety of pre-trained transform-based models.

According to BERT and AraBERT, we decided to utilize Adam, the optimizer with a learning rate of 2e-5. These two models produce the greatest results and the quickest when compared to the other optimizers [24], [25], [26]. For the loss function, we went with the Cross Entropy to evaluate how well a classification model performs when its output is a number between 0 and 1.

3.4 Formulations of the performance metrics

The main evaluation metrics that are adapted to measure the efficiency and performance of our proposed ArSa-Tweets model are described as follows [27], [28], [29]:1. Accuracy: is the fraction from computing the number of correct tweets related to the total count of all input tweets. The mathematical formula of Accuracy is in Eq. (1).(1) Accuracy=TP+TNTP+FP+TN+FN

where TP represents the true positive, TN implies to true negative, FP implies to the false positive, and the FN represents the false negative.

2. Precision: is calculated to find the relevant tweets ratio among the obtained correct tweets, as shown in Eq. (2).(2) Precision=TPTP+FP

3. Recall: is the percentage of the retrieved tweets for both TP and TN, as explained in Eq. (3).(3) Recall=TPTP+TN

4. F1_score: is the main one, which is considered the harmonic metric that means precision and recall, as illustrated in Eq. (4).(4) F1_Score=2Recall×PrecisionRecall+Precision

3.5 Reproducibility of the proposed model

This section describes the strategy of using Deep Learning models that are deployed in our proposed system for detecting sarcastic sentences. We basically used LSTM, Multi-headed CNN-LSTM-GRU, BERT, AraBert-V01, and AraBert-V02 as baseline models, to develop our proposed ArSa-Tweets detection system, that will be discussed in detail at Section 3.6.

Regarding the tweet as a collection of words, LSTM and BERT offer the benefit of helping recall long-term special and temporal dependencies by establishing a connection between past and current contexts [30]. Then, we examined the Multi-headed-LSTM-CNN-GRU Model, and we used AraBERT by utilizing word embeddings to represent each token in the tweet by its corresponding contextualized embeddings. So, we report the effect of classifiers in predicting the matching general feelings. In order to highlight certain words and extract the most important portion of a target sentence, we employed the attention mechanism. Notably, the features extracted technique (PCA) with the effective preprocessing are joined to attain the last representation of sentential information. As a result, this technique will enhance the classifiers' performance. So, the following explains the proposed preprocessing system.

3.5.1 Proposed preprocessing phase

Before feeding the tweets into the models, we perform different preprocessing processes on them. Primarily, the proposed preprocessing phase includes; i) text normalization to eliminate the Arabic diacritic marks “Al Tashkeel Marks Image 4” by the “pyarabic” library in Python, any non-Arabic character, and tap. ii) Replacing all samples of “Image 5” with “Image 6”, “Image 7” with “Image 8”, and Image 9 with Image 10. iii) Replacing any hyperlink with “URLs” words. iv) Striping all diacritics (if any). v) Eliminating new lines and integrating each sentence into a single text. vi) Eliminating Arabic stop words and punctuation from the tweets that do not add any more context. To ensure text uniformity, we will also convert any inconsistent language formatting to standard Arabic alphabets. We segmented the input sentence using the AraBERT model in order to extract each word and its parts. Using AraBERT improves the classifier's comprehension, which improves performance, as related studies demonstrated.

The contextual data is then translated into numerical values in the following stage. This stage uses a TF-IDF technique in combination with a pre-trained word embedding system. Based on the most common terms in the dataset, the generated word embedding produces 2,000 features for every sample. At the end of this stage, every word is assigned a one-hot vector.

Capturing the most variable data components of samples is possible with Principal Component Analysis (PCA), making it an effective technique for representing data [31]. Applications of the principle components analysis methodology include machine learning [32], face recognition [33], and image denoising [34], [35]. Additionally, when researchers tried some novel techniques, PCA served as a baseline approach. However, practically all of this PCA research concentrates on the technology's usage in feature extraction.

So, the PCA was employed in order to lower the computing complexity of the procedure [36]. Using PCA, one can determine which axes (or directions) in a dataset best represent its variability. To achieve this, it first determines which directions in the data have the greatest variation, after which the data is projected onto the new axes. PCA can also be used to identify a second axis that is orthogonal to the first one and explains most of the remaining variation in the dataset. This axis can be used to determine how much variation is captured in the training set. We established the 99% criterion according to [37] for reducing the input dataset's dimension in order to make sure that we are employing 99% of relevant information in this effort. Using PCA reduces the ArSa-data feature set from 3000 features to 2,727 features.

Before the training process, the used data is divided into 10% for testing data and 90% for training data. For the experimental test, we deployed the ‘Hold-Out’ validation technique, which is adapted from [38]. As such, the training data again is divided into 10% for validation, and 90% for training. So, the overall dataset is divided into the following: the testing data has 300 tweets, the training data consists of 2,430 tweets, and the remaining data for validation consists of 270 tweets. The main purpose of a small proportion (10%) of the validation set is to keep each model from overfitting, and the model can generalize well on unseen data. Whereas, data testing is mainly adapted for computing the performance and the efficiency of the proposed model. Fig. 3 displays an overview of all preprocessing and feature extraction procedures.Figure 3 Architectures of preprocessing and feature extraction. The proposed preprocessing phase contains Text normalization, Swapping, and Removal. Then, the contextual data is then translated into numerical values by a TF-IDF technique in combination with a pre-trained word embedding system. Principal Component Analysis (PCA) is employed to lower the computing complexity of the procedure. Finally, the ‘Hold-Out’ validation technique is adopted to divide tweets into training, validation, and testing data.

Figure 3

3.6 Baseline methods

3.6.1 LSTM model

Long-Short Term Memory (LSTM) is a new enhanced version of Recurrent Neural Network (RNN) [39], [40]. LSTM comes to transform RNN status as confirming what needs to be preserved and discarding what needs to be removed. Further, LSTM is generated by raising the power of RNNs memory [41]. While normal RNN suffers from issues of exploding and vanishing gradient, therefore LSTM can tackle such issues. Also, it is generated to learn long-distance dependencies through consecutive instances. By using deeper layers, LSTM preserves the value of the loss function to back-propagate, where the training route continues for several steps. Every LSTM unit holds many gates like input, forget, and output gates which are responsible for arranging and choosing by the ratio of data to maintain, discard, and transfer to the next round. Moreover, it is responsible for specifying either passing or blocking information movement over the LSTM unit. In specific, the output gate creates the current hidden state and output by combining a Sigmoid function with a ‘tanh’ function, while the input gate and forget gate employ Sigmoid functions to determine how much information to allow into or out of the memory cell. Fig. 4 demonstrates the architecture of LSTM, which works in feed-forward neural network architecture with multi-layer, so each gate of LSTM can be thought and act as a standard neuron to calculate the weighted sum based on activation function. The following mathematical formulas in Equations (5)-(10) are used to calculate the input, forget, and output, where the output gate in these levels can be joined with the input gate as represented in Eq. (10).(5) it=σ(Wiyxt+Wizht−1+bi)

(6) ft=σ(Wfyxt+Wfzht−1+bf)

(7) ot=σ(Woyxt+Wozht−1+bo)

(8) dt=σ(Wdyxt+Wdzht−1+bd)

(9) ct=ft⊗ct−1+it⊗dt

(10) ht=tanh(Ct)⊗Ot

where σ refers to the activation function (Sigmoid function). The Wi, Wf, Wo, and Wd represent the weighting factors that are used to map the main gates with hidden layers at the input step. The ⊗ presents element multiplication. ht=[ht−n...ht−1] represents the output of the final LSTM layer.Figure 4 Detail representation of LSTM model, some parts adapted from [42], [43]. The input (it) alters the state of the LSTM cell, and the gate (gt) works to update as a candidate memory, which decides the potential new cell states. The forgets gate (ft) resets the memory and output of the cell, and the output gate (ot) decides the output candidates. While the Xt represents the input data, and the ht = [ht−n...ht−1] represents the output of the final LSTM layer. The Ct = [Ct−n...Ct−1] presents the joining between the input and the output by addition (⊕) or multiplication (⊗). The tanh function is applied on the output gate to generate the vectors on the cell.

Figure 4

3.6.2 Multi-headed-LSTM-CNN-GRU model

Multi-headed-LSTM-CNN-GRU is another version of the LSTM model that comprises two different Bidirectional models of LSTM-CNN and GRU-CNN, in addition to the CNN model, which was developed by [44]. Initially, we used the Arabic news embedding from [45], [46] for a text representation, and the ‘emoji2vec’ from [47] is used for emoji data. After that, these embeddings are fed to this ‘Multi-headed-LSTM-CNN-GRU’ deep-learning model. The first step in training starts by inputting the processed clean text and emoji into the network. Meanwhile, the processing of clean text is done in both Bidirectional models (LSTM-CNN and GRU-CNN). The intermediate results are combined and pushed through the batching norm layer, and then the output is also inputted into the dense layer. On the other hand, the processing of emojis is done by the CNN-LSTM model. Following this, the outputs of the emoji layer are combined with the dense layer. In the current stage, the final results are inputted into two fully connected layers.

3.6.3 BERT model

Multi-layer Transformer encoder was first employed by [48], [49] as a basic architecture to form the BERT model. BERT Transformer, which relied on utilizing a bidirectional self-attention process, was then presented in [50]. The bidirectional technique eliminates the restrictions of self-attention, which limits incorporating the text from the left or right side only. Unlike what was done in the Word2Vec model that was presented by [51] for creating, and embedding the input in the BERT model is not a vector of words representation. In contrast, the input comprises the embedding of token, segment, and position. Note that the WordPiece embedding of 30k tokens forms the BERT token embedding [49].

Two unsupervised tasks have been used to pre-train the BERT in our proposed model as follows: i) Masked Language Model (LM): this task forecasts the amount of arbitrary masked tokens found in the input to train the bidirectional encoder. ii) Next Sentence Prediction (NSP): this task forecasts the next sentence of the input sentence to recognize sentence associations. Thus, the pre-trained BERT model can perform better training for various NLP tasks, where sentence relations are also vital for question-answering (QA), and Natural Language Inference (NLI) tasks.

The reason behind using the BERT model here for text classification is the ability of the fine-tuning technique to be used by deploying the Softmax layer on the BERT network for classification tasks, as seen in Eq. (11). The model is also used to classify c, which is a sequence of text into a proper category.(11) p(c|H)=Softmax(WH)

where H represents the last hidden layer output from the first token. Then, H is used as the next input to the layer of Softmax. W represents the main parameter of the classification layer. To increase the log probability of the correct label, these W parameters will be fine-tuned with the whole parameters of the BERT model.

3.6.4 AraBERT-v1 and AraBERT-v2

AraBERT is identified as another developed model from BERT, which was basically pre-trained for the Arabic language [53]. Around 70 Million sentences with 3 Billion words have been used for training purposes. The used dataset in the training step is a pool of publicly accessible large-scale Arabic contexts. There are two fine-tuning versions of AraBERT, which are known as AraBERT-v1 and AraBERT-v2.

Fig. 5 represents the architecture of AraBERT, which manipulates tweets in two sections. Using token embedding, the entry tweet is embedded. Meanwhile, the embedding process and placement embedding start from bottom to top. AraBERT transformer acts as an encoder, where every section of the tweet is transformed and saved in a vector. The opposite transformer acts as a decoder and is used based on the activation function to calculate the final score of every section.Figure 5 Architecture of AraBERT [52]. AraBERT splits tweets into two parts. The entry tweet is embedded via token embedding. Concurrently, the placement embedding and embedding procedure begin from the bottom up. Each portion of the tweet is converted and saved as a vector using the AraBERT transformer (encoder). The ultimate score for each segment is determined by using the opposite transformer (decoder).

Figure 5

4 Experimental results

This section reports the obtained outputs from our proposed ArSa-Tweet model compared to the obtained results from other competitive models in the literature.

4.1 Experimental results of five models

Table 3 reports the enhancement in outcomes for the five developed DL-based models before and after conducting our proposed preprocessing steps using the proposed “ArSa-data” as a golden corpus that was discussed in Section 3.5.1. We ensure convergence of the models when the network's error, as indicated by the loss function, no longer decreases noticeably concerning the number of iterations (or epochs), meaning that the network's parameters are no longer getting better. The findings reported in Table 3 showed that BERT, AraBert-V01, and AraBert-V02 are the best performance models, respectively.Table 3 Results obtained from proposed model deployed based on five DL models using our proposed ArSa-data corpus.

Table 3Metrics	Prepossessing	LSTM	Multi-headed-	BERT	AraBert-V01	AraBert-V02	
LSTM-CNN-GRU	
Accuracy	Yes	77.00%	76.34%	85.67%	86.67%	87.00%	
No	76.20%	75.40%	84.47%	83.58%	83.83%	


	
Precision	Yes	76.80%	75.48%	88.03%	86.70%	87.00%	
No	75.43%	75.78%	84.27%	83.60%	83.80%	


	
Recall	Yes	76.80%	71.18%	78.03%	86.70%	87.00%	
No	75.43%	73.53%	80.65%	83.60%	83.80%	


	
F1_score	Yes	76.80%	73.11%	82.73%	86.70%	87.00%	
No	75.43%	74.54%	82.42%	83.60%	83.80%	

In a nutshell, the best performance is enhanced from (approximately 83.80%) to (87%) and from (approximately 83.6%) to (approximately 86.7%) for all metrics (Accuracy, Precision, Recall, and F1_score) based on AraBert-V01 and AraBert-V02 models, respectively. While based on adapting the BERT model, the accuracy metrics are enhanced from (84.47%, 84.27%, and 82.42%) to (85.67%, 88.03%, and 82.73%) for the Accuracy, Precision, and F1_score. In contrast, the Recall metric decreased from 80.65% to 78.03%.

Fig. 6 shows the confusion matrix of applying BERT model. It correctly classified 103 out of 132 sarcastic tweets. Also, the model correctly classified 154 out of 168 non-sarcastic tweets.Figure 6 Confusion Matrix on ArSa-Tweets dataset when using BERT model. The proposed model correctly classified 103 out of 132 sarcastic tweets, and it correctly classified 154 out of 168 non-sarcastic tweets.

Figure 6

Fig. 7 shows the confusion matrix of the AraBert-V01 model. It indicates that 77 out of 106 sarcastic tweets were correctly classified, while 183 out of 194 non-sarcastic tweets were correctly classified.Figure 7 Confusion Matrix on ArSa-Tweets dataset when using AraBert-V01 model. The proposed model correctly classified 77 out of 106 sarcastic tweets, and it correctly classified 183 out of 194 non-sarcastic tweets.

Figure 7

In Fig. 8, the confusion matrix of the AraBert-V02 model indicates that 79 out of 106 sarcastic tweets were correctly classified, while 182 out of 194 non-sarcastic tweets were correctly classified.Figure 8 Confusion Matrix on ArSa-Tweets dataset when using AraBert-V02 model. Our proposed model correctly classified 77 out of 106 sarcastic tweets, and it correctly classified 182 out of 194 non-sarcastic tweets.

Figure 8

According to the overall results, the suggested system can handle sarcasm identification in the tweet domain. It shows the effectiveness of the deep neural networks when combined with the appropriate preprocessing algorithms. The outcome demonstrates that our ArSa-Tweet model can learn contextual information from the sarcastic expression and make extensive use of it by improving the model's performance. Furthermore, word sense and contextual information in sarcastic expressions can be captured by the PCA technique, and can help in feature extraction phases that are frequently encountered in deep learning techniques.

4.2 Comparative results between proposed models and other related works

In this section, we conduct new wide experiments to compare our proposed ArSa-Tweets model against other state-of-the-art methods based on using different datasets. Three new different sarcasm datasets are used as main benchmarks, which are ArSarcasm-v1, ArSarcasm-v2, and SemEval-2022. These other corpora are previously published and used in [7], [54]. So, we trained our five developed DL-based models over these three corpora to confirm the robustness of our proposed system in improving the final accuracy and performance. Table 4 has the obtained accuracy results using these three sarcasm datasets. This evaluation process makes a new benchmark comparison between our proposed work that is conducted based on our proposed preprocessing with cutting edge of deploying five DL-based models; and comparing it with other previous models published in [7], [55], [56], [57], [58], [59], [60].Table 4 A Benchmark comparison between our proposed ArSa-Tweets model based on deploying preprocessing steps with the five models against other related works.

Table 4Dataset	Models	Accuracy	Precision	Recall	F1_score	
ArSarcasm-v1[7]	BiLSTM [7]	-	62.00%	38.00%	46.00%	
XLM-RBASE, XLM-RLARGE [55]	-	-	-	73.22%, 73.72%	
LSTM (Ours)	84.76%	84.80%	84.80%	84.80%	
Multi-headed-LSTM-CNN-GRU (Ours)	81.69%	80.26%	76.39%	78.16%	
BERT (Ours)	87.50%	61.54%	44.44%	51.61%	
AraBert-V01 (Ours)	86.16%	86.20%	86.20%	86.20%	
AraBert-V02 (Ours)	85.80%	85.80%	85.80%	85.80%	
	
ArSarcasm-v2[54]	AraBERTv2-large [57]	86.37%	85.16%	86.37%	85.07%	
MTL ATTINTER [58]	86.61%	81.06%	76.60%	78.46%	
RF, AraBERT [6]	-	68.58%	67.00%	51.89%	
LSTM, Aravec [61]	-	89.7%	42.5%	48.6%	
Multi-headed-LSTM-CNN-GRU, MARBERT [44]	-	72.31%	70.04%	56.62%	
AraBERT, Sentence BERT [62]	-	72.68%	72.35%	59.89%	
LSTM (Ours)	83.00%	82.90%	82.90%	82.90%	
Multi-headed-LSTM-CNN-GRU (Ours)	81.69%	78.16%	79.14%	78.54%	
BERT (Ours)	85.58%	70.91%	19.12%	30.12%	
AraBert-V01 (Ours)	85.18%	85.20%	85.20%	85.20%	
AraBert-V02 (Ours)	85.93%	85.90%	85.90%	85.90%	
	
SemEval-2022[7]	BERT-base-cased [59]	86.00%	-	-	86.00%	
Three deep learning-based models [60]	-	-	-	62.17%	
SarcasmDet [63]	69.57%	62.40%	74.1%	61.14%	
LSTM (Ours)	75.83%	75.80%	75.80%	75.80%	
Multi-headed-LSTM-CNN-GRU (Ours)	77.77%	77.00%	74.03%	75.39%	
BERT (Ours)	87.46%	74.36%	75.32%	74.84%	
AraBert-V01 (Ours)	86.54%	86.50%	86.50%	86.50%	
AraBert-V02 (Ours)	86.70%	86.70%	86.70%	86.70%	

Table 4 reports the comparative results in the following five terms, which are the dataset used, the models used in the published papers, and the last four terms, which are the Accuracy, Precision, Recall, and F1_score metrics.

The evaluation results listed in Table 4 can be discussed and summarized as follows:1. Our proposed ArSa-Tweets model obtained the best performance results when conducting the adapted AraBert-V01 model using ArSarcasm-v1 dataset in all metrics [7], which reached 86.2% in F1_score. Moreover, our proposed model produced a more potent performance when compared with other works published in [7], [55]. While, these previous works achieved F1_score 46% and 73.72%, respectively.

2. For ArSarcasm-v2 data collected by [54], the best performance is achieved when deploying LSTM, AraBert-V01, and AraBert-V02 in our proposed model, which outperformed the other work published in [60]. Meanwhile, according to [57], our proposed model obtained good results in the Precision and F1_score metrics when using the AraBert-V02 model.

3. For SemEval-2022 data, our proposed system got the highest results based on both versions of AraBert-V01 and AraBert-V01, when comparing with [59], [60], which was reported for iSarcasmEval (as Sub-task A for Arabic corpus) competition with [64]. As such, the results reached 86.50% and 86.70% in the two versions of AraBert, respectively. While the best outcomes are obtained by the AraBert model, unfortunately BERT model reported the worst results in Precision, Recall, and F1_Score, which reached 74.36%, 75.32%, and 74.84%, respectively.

Finally, to determine whether there was a statistically significant difference between the results generated by our method and other algorithms mentioned in this study, the non-parametric Friedman test was employed [65], [28]. In this test, there are two distinct hypotheses: the alternative hypothesis (H1) and the null hypothesis (H0). In the experiments, H0 means: “There are no variations in performance across any algorithms used for a given metric”. H1 means: “There are variations in performance across any algorithms used for a given metric”. Table 5 displays the findings of the investigation using the Friedman test. Determining the alpha value was 0.05, and determining the degrees of freedom (Df) (number of samples compared -1) was 7. Based on the degrees of freedom and p-value, H1 is approved whereas H0 is refused. In other words, the Friedman test indicates a statistically significant difference between our methods and others of compared algorithms.Table 5 Evaluation of the findings with a Friedman test in terms of F1-score.

Table 5Details	ArSarcasm-v1	ArSarcasm-v2	SemEval-2022	
Degrees of freedom	2	3	3	
W	1	1	0.917	
p-value	2.061154e-09	5.878231e-13	6.799043e-12	
Conclusion	Reject H0	Reject H0	Reject H0	

5 Conclusion

This paper proposes a new empowered DL-based tweet detection method for Arabic sarcasm called ArSa-Tweets by developing five different deep-learning models. A new robust preprocessing phase is invited to improve the performance of our proposed sarcasm detection model. Further, this work presented a new golden corpus called ArSa-data to support sarcasm analysis of Arabic Twitter, which is considered to be a publicly sarcastic corpus for researchers in this domain. We gathered this sarcastic corpus which includes 3,000 tweets. The data then is manually annotated by two expert annotators. Baseline experiments are conducted by deploying five NLP models: LSTM, Multi-headed CNN-LSTM-GRU, BERT, AraBert-V01, and AraBert-V02. To prove the work proposed Arabic sarcasm detection system, several preprocessing steps are conducted in the proposed ArSa-Tweets system to enhance the performance. Also, we examined the proposed preprocessing with the five cutting-edge models using three shared previous datasets. These experiments can be considered as a benchmark for further work on sarcastic Arabic tweets. As demonstrated in this paper and preceding works, sarcasm annotation is the most sophisticated and complicated task. However, these challenges are highlighted, which could help as a practical guide for upcoming annotation tasks of Arabic sarcasm analysis.

In addition, governments, and decision-makers can profit greatly from the present effort in monitoring people's emotions or opinions over material in social media. Additionally, the goal of our present study is to identify potential irony by assessing people's emotions following sarcastic posts, and to develop company tactics based on the emotions of customers.

In the future, we will assess various algorithms, with different prepossessing functions, to ensure that they can also enhance the performance by detecting the sarcastic text in tweets. To guarantee accurate results, the suggested approach will also be tested on a bigger, better-structured Arabic corpus. Toward this aim, we want to gather a well-structured dataset using the current standard Arabic dialectic to guarantee that the suggested approach yields excellent performance in differentiating between hostile, abusive, and normal settings. In addition to making our ArSa-data corpus publicly available to the researcher's community, we aim to create a web-based irony recognizer that can scan tweets, weed out negative posts, and instantly identify the sarcastic label.

CRediT authorship contribution statement

Qusai Abuein: Writing – original draft, Validation, Resources, Methodology, Funding acquisition, Formal analysis, Data curation. Ra'ed M. Al-Khatib: Writing – review & editing, Writing – original draft, Visualization, Supervision, Software, Project administration, Methodology, Formal analysis, Conceptualization. Aya Migdady: Writing – original draft, Software, Resources, Project administration, Methodology, Investigation, Data curation, Conceptualization. Mahmoud S. Jawarneh: Validation, Project administration, Methodology, Funding acquisition, Formal analysis, Conceptualization. Asef Al-Khateeb: Writing – review & editing, Writing – original draft, Validation, Resources, Project administration, Funding acquisition, Formal analysis, Conceptualization.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
==== Refs
References

1 Liu B. Sentiment analysis and opinion mining Synth. Lect. Hum. Lang. Technol. 5 1 2012 1 167
2 Joshi A. Bhattacharyya P. Carman M.J. Automatic sarcasm detection: a survey ACM Comput. Surv. 50 5 2017 1 22
3 Wilson D. The pragmatics of verbal irony: echo or pretence? Lingua 116 10 2006 1722 1743
4 Bouazizi M. Ohtsuki T.O. A pattern-based approach for sarcasm detection on Twitter IEEE Access 4 2016 5477 5488
5 Khodak M. Saunshi N. Vodrahalli K. A large self-annotated corpus for sarcasm arXiv preprint arXiv:1704.05579 2017
6 Elgabry H. Attia S. Abdel-Rahman A. Abdel-Ate A. Girgis S. A contextual word embedding for Arabic sarcasm detection with random forests Proceedings of the Sixth Arabic Natural Language Processing Workshop 2021 340 344
7 Farha I.A. Magdy W. From Arabic sentiment analysis to sarcasm detection: the arsarcasm dataset Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection 2020 32 39
8 Rosenthal S. Farra N. Nakov P. Semeval-2017 task 4: sentiment analysis in Twitter, arXiv preprint arXiv:1912.00741 2019
9 Davidov D. Tsur O. Rappoport A. Semi-supervised recognition of sarcasm in Twitter and Amazon Proceedings of the Fourteenth Conference on Computational Natural Language Learning 2010 107 116
10 Al-Khatib R.M. Al-Betar M.A. Awadallah M.A. Nahar K.M. Shquier M.M.A. Manasrah A.M. Doumi A.B. MGA-TSP: modernised genetic algorithm for the travelling salesman problem Int. J. Reason.-Based Intell. Syst. 11 3 2019 215 226
11 Ranasinghe T. Saadany H. Plum A. Mandhari S. Mohamed E. Orasan C. Mitkov R. Rgcl at Idat: Deep Learning Models for Irony Detection in Arabic Language 2019
12 Khalifa M. Hussein N. Ensemble learning for irony detection in Arabic tweets FIRE (Working Notes) 2019
13 Al-Khatib R.M. Zerrouki T. Abu Shquier M.M. Balla A. Al-Khateeb A. A new enhanced Arabic light stemmer for IR in medical documents Comput. Mater. Continua 68 1 2021 1255 1269
14 Ra'ed M A.-K. Al-qudah N.E.A. Jawarneh M.S. Al-Khateeb A. A novel improved lemurs optimization algorithm for feature selection problems J. King Saud Univ, Comput. Inf. Sci. 35 8 2023 101704
15 Ameur M.S.H. Aliane H. Aracovid19-ssd: Arabic covid-19 sentiment and sarcasm detection dataset arXiv preprint arXiv:2110.01948 2021
16 Nahar K.M. Al-Khatib R.M. Al-Shannaq M. Daradkeh M. Malkawi R. Direct text classifier for thematic Arabic discourse documents Int. Arab J. Inf. Technol. 17 3 2020
17 Abdelaal M.A. Fattah M.A. Arafa M.M. Predicting sarcasm and polarity in Arabic text automatically: supervised machine learning approach J. Theor. Appl. Inf. Technol. 100 8 2022
18 Ameur A. Hamdi S. Yahia S.B. Domain adaptation approach for Arabic sarcasm detection in hotel reviews based on hybrid learning Proc. Comput. Sci. 225 2023 3898 3908
19 Nahar K.M. Al-shannaq M. Alshorman R. Al-Khatib R.M. tom M.A.Ot. Handicapped wheelchair movements using discrete Arabic command recognition Sci. J. King Faisal Univ. (Basic Appl. Sci.) 21 1 2020 171 184
20 Galal M.A. Yousef A.H. Zayed H.H. Medhat W. Arabic sarcasm detection: an enhanced fine-tuned language model approach Ain Shams Eng. J. 2024 102736
21 Aleryani G.H. Deabes W. Albishre K. Abdel-Hakim A.E. Impact of emoji exclusion on the performance of Arabic sarcasm detection models arXiv preprint arXiv:2405.02195 2024
22 Rahma A. Azab S.S. Mohammed A. A comprehensive review on Arabic sarcasm detection: approaches, challenges and future trends IEEE Access 2023
23 Alaiad A. Migdady A. Al-Khatib R.M. Alzoubi O. Zitar R.A. Abualigah L. Autokeras approach: a robust automated deep learning network for diagnosis disease cases in medical images J. Imag. 9 3 2023 64
24 Rashaideh H. Sawaie A. Al-Betar M.A. Abualigah L.M. Al-Laham M.M. Al-Khatib R.M. Braik M. A grey wolf optimizer for text document clustering J. Intell. Syst. 29 1 2020 10.1515/jisys-2018-0194
25 Al-Khatib R.M. Abdullah R. Abdul Rashid N. A comparative taxonomy of parallel algorithms for RNA secondary structure prediction Evol. Bioinform. 2010 6 2010 27 45
26 Al-Khatib R.M. Zerrouki T. Abu Shquier M.M. Balla A. Tashaphyne0. 4: a new Arabic light stemmer based on rhyzome modeling approach Inf. Retr. J. 26 1 2023 14
27 Rustam F. Ashraf I. Mehmood A. Ullah S. Choi G.S. Tweets classification on the base of sentiments for us airline companies Entropy 21 11 2019 1078
28 Zamri N.E. Azhar S.A. Mansor M.A. Alway A. Kasihmuddin M.S.M. Weighted random k satisfiability for k= 1, 2 (r2sat) in discrete hopfield neural network Appl. Soft Comput. 126 2022 109312
29 Pimpalkar A. Raj R.J.R. Evaluation of tweets for content analysis using machine learning models 2020 12th International Conference on Computational Intelligence and Communication Networks (CICN) 2020 IEEE 454 459
30 Nahar K.M. Al-Khatib R.M. Al-Shannaq M.A. Barhoush M.M. An efficient holy Quran recitation recognizer based on SVM learning model Jordanian J. Comput. Inf. Technol. (JJCIT) 6 04 2020
31 Xu Y. Zhang D. Yang J.-Y. A feature extraction method for use with bimodal biometrics Pattern Recognit. 43 3 2010 1106 1115
32 Sudharsan M. Thailambal G. Alzheimer's disease prediction using machine learning techniques and principal component analysis (pca) Mater. Today Proc. 81 2023 182 190
33 Regin R. Rajest S.S. Shynu T. Principal component analysis for atm facial recognition security Cent. Asian J. Med. Nat. Sci. 4 3 2023 292 311
34 Doniza L. Lee M. Katzir T.B. Artzi M. Bashat D.B. Radunsky D. Shmueli K. Ben-Eliezer N. Noise propagation and mp-pca image denoising for high-resolution quantitative t2* and magnetic susceptibility mapping (qsm) arXiv preprint arXiv:2404.19309 2024
35 Nahar K.M. Abul-Huda B. Abu Naser A. Al-Khatib R.M. Twins and similar faces recognition using geometric and photometric features with transfer learning Int. J. Comput. Digit. Syst. 2021 10.12785/ijcds/110110
36 Beattie J.R. Esmonde-White F.W. Exploration of principal component analysis: deriving principal component analysis visually using spectra Appl. Spectrosc. 75 4 2021 361 375 33393349
37 Rippel O. Mertens P. Merhof D. Modeling the distribution of normal data in pre-trained deep features for anomaly detection 2020 25th International Conference on Pattern Recognition (ICPR) 2021 IEEE 6726 6733
38 Ripley B.D. Pattern Recognition and Neural Networks 2007 Cambridge University Press
39 Zen H. Agiomyrgiannakis Y. Egberts N. Henderson F. Szczepaniak P. Fast, compact, and high quality lstm-rnn based statistical parametric speech synthesizers for mobile devices arXiv preprint arXiv:1606.06061 2016
40 Graves A. Schmidhuber J. Framewise phoneme classification with bidirectional lstm and other neural network architectures Neural Netw. 18 5–6 2005 602 610 16112549
41 Salehinejad H. Sankar S. Barfett J. Colak E. Valaee S. Recent advances in recurrent neural networks arXiv preprint arXiv:1801.01078 2017
42 Eke C.I. Norman A.A. Shuib L. Context-based feature technique for sarcasm identification in benchmark datasets using deep learning and bert model IEEE Access 9 2021 48501 48518
43 Aggarwal S. The ultimate guide to building your own lstm models https://www.projectpro.io/article/lstm-model/832 2024
44 Abdel-Salam R. Wanlp 2021 shared-task: towards irony and sentiment detection in Arabic tweets using multi-headed-lstm-cnn-gru and marbert Proceedings of the Sixth Arabic Natural Language Processing Workshop 2021 306 311
45 Altowayan A.A. Elnagar A. Improving Arabic sentiment analysis with sentiment-specific embeddings 2017 IEEE International Conference on Big Data (Big Data) 2017 IEEE 4314 4320
46 Altowayan A.A. Tao L. Word embeddings for Arabic sentiment analysis 2016 IEEE International Conference on Big Data (Big Data) 2016 IEEE 3820 3825
47 Eisner B. Rocktäschel T. Augenstein I. Bošnjak M. Riedel S. emoji2vec: learning emoji representations from their description arXiv preprint arXiv:1609.08359 2016
48 Vaswani A. Shazeer N. Parmar N. Uszkoreit J. Jones L. Gomez A.N. Kaiser Ł. Polosukhin I. Attention is all you need Adv. Neural Inf. Process. Syst. 30 2017
49 Wu Y. Schuster M. Chen Z. Le Q.V. Norouzi M. Macherey W. Krikun M. Cao Y. Gao Q. Macherey K. Google's neural machine translation system: bridging the gap between human and machine translation arXiv preprint arXiv:1609.08144 2016
50 Devlin J. Chang M.-W. Lee K. Toutanova K. Bert: pre-training of deep bidirectional transformers for language understanding arXiv preprint arXiv:1810.04805 2018
51 Mikolov T. Sutskever I. Chen K. Corrado G.S. Dean J. Distributed representations of words and phrases and their compositionality Adv. Neural Inf. Process. Syst. 26 2013
52 Elfaik H. Combining context-aware embeddings and an attentional deep learning model for Arabic affect analysis on Twitter IEEE Access 9 2021 111214 111230
53 Antoun W. Baly F. Hajj H. Arabert: transformer-based model for Arabic language understanding arXiv preprint arXiv:2003.00104 2020
54 Farha I.A. Zaghouani W. Magdy W. Overview of the wanlp 2021 shared task on sarcasm and sentiment detection in Arabic Proceedings of the Sixth Arabic Natural Language Processing Workshop 2021 296 305
55 Khalifa M. Hassan H. Fahmy A. Zero-resource multi-dialectal Arabic natural language understanding arXiv preprint arXiv:2104.06591 2021
56 Alkhateeb F. Al-Khatib R.M. Doush I.A. A survey for recent applications and variants of nature-inspired immune search algorithm Int. J. Comput. Appl. Technol. 63 4 2020 354 370
57 Wadhawan A. Arabert and farasa segmentation based approach for sarcasm and sentiment detection in Arabic tweets arXiv preprint arXiv:2103.01679 2021
58 Mahdaouy A.E. Mekki A.E. Essefar K. Mamoun N.E. Berrada I. Khoumsi A. Deep multi-task model for sarcasm detection and sentiment analysis in Arabic language arXiv preprint arXiv:2106.12488 2021
59 Díaz P.G. Cordon P. Mata J. Pachón V. I2c at semeval-2022 task 6: intended sarcasm detection on social networks with deep learning Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022) 2022 877 880
60 Mahdaouy A.E. Mekki A.E. Essefar K. Skiredj A. Berrada I. Cs-um6p at semeval-2022 task 6: transformer-based models for intended sarcasm detection in English and Arabic arXiv preprint arXiv:2206.08415 2022
61 Ghoul D. Lejeune G. Sarcasm and sentiment detection in Arabic: investigating the interest of character-level features Proceedings of the Sixth Arabic Natural Language Processing Workshop 2021 329 333
62 Bashmal L. AlZeer D. Arsarcasm shared task: an ensemble bert model for sarcasmdetection in Arabic tweets Proceedings of the Sixth Arabic Natural Language Processing Workshop 2021 323 328
63 Abdullah M. Alnore D. Swedat S. Khrais J. Al-Ayyoub M. Sarcasmdet at semeval-2022 task 6: detecting sarcasm using pre-trained transformers in English and Arabic languages Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022) 2022 1025 1030
64 Farha I.A. Oprea S.V. Wilson S. Magdy W. Semeval-2022 task 6: isarcasmeval, intended sarcasm detection in English and Arabic Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022) 2022 802 814
65 Zamri N.E. Mansor M.A. Kasihmuddin M.S.M. Sidik S.S. Alway A. Romli N.A. Guo Y. Jamaludin S.Z.M. A modified reverse-based analysis logic mining model with weighted random 2 satisfiability logic in discrete Hopfield neural network and multi-objective training of modified niched genetic algorithm Expert Syst. Appl. 240 2024 122307
