List of Articles BERT

• Open Access Article
  • Abstract Page
  • Full-Text
  
  1 - On the Behavior of Pre-trained Word Embedding Variants in Deep Headline Generation from Persian Texts
  Mohammad Ebrahim Shenassa Behrooz Minaei-Bidgoli
  Inspired by sequence-to-sequence models for machine translation, deep-learning based summarization methods were presented. The summaries generated this way, are structurally more readable and usually convey the complete meaning to the reader. In these methods, embeddi More

  Inspired by sequence-to-sequence models for machine translation, deep-learning based summarization methods were presented. The summaries generated this way, are structurally more readable and usually convey the complete meaning to the reader. In these methods, embedding vectors are used for semantic representation, in which the weight of each word vector is learned according to its neighboring words from a large corpus. In static word embedding, the weight of the vectors is obtained by choosing a proximity window for each word. But in contextual ones like BERT, multilayer transformers are applied to calculate the weight of these vectors, which pay attention to all the words in the text. So far, several papers have shown that contextual word embedding are more successful than the other ones due to the ability of fine-tuning the weights to perform a specific natural language processing task. However, the performance of the initial weights of these vectors is not investigated for headline generation from Persian texts. In this paper, we will investigate the behavior of pre-trained word embedding variants without fine-tuning in deep headline generation from Persian texts. To train the headline generation model, "Elam Net" is used, which is a Persian corpus containing about 350 thousand pairs of abstracts and titles of scientific papers. The results show that the use of BERT model, even without fine-tuning its weights, is effective in improving the quality of generated Persian headlines, bringing the ROUGE-1 metric to 42%, which is better than the other pre-trained ones. Manuscript profile
• Open Access Article
  • Abstract Page
  • Full-Text
  
  2 - Ranking Improvement Using BERT
  shekoofe bostan Ali-Mohammad Zare-Bidoki Mohammad-Reza Pajoohan
  In today's information age, efficient document ranking plays a crucial role in information retrieval systems. This article proposes a new approach to document ranking using embedding models, with a focus on the BERT language model to improve ranking results. The propose More

  In today's information age, efficient document ranking plays a crucial role in information retrieval systems. This article proposes a new approach to document ranking using embedding models, with a focus on the BERT language model to improve ranking results. The proposed approach uses vocabulary embedding methods to represent the semantic representations of user queries and document content. By converting textual data into semantic vectors, the relationships and similarities between queries and documents are evaluated under the proposed ranking relationships with lower cost. The proposed ranking relationships consider various factors to improve accuracy, including vocabulary embedding vectors, keyword location, and the impact of valuable words on ranking based on semantic vectors. Comparative experiments and analyses were conducted to evaluate the effectiveness of the proposed relationships. The empirical results demonstrate the effectiveness of the proposed approach in achieving higher accuracy compared to common ranking methods. These results indicate that the use of embedding models and their combination in proposed ranking relationships significantly improves ranking accuracy up to 0.87 in the best case. This study helps improve document ranking and demonstrates the potential of the BERT embedding model in improving ranking performance. Manuscript profile
• Open Access Article
  • Abstract Page
  • Full-Text
  
  3 - Applying deep learning for improving the results of sentiment analysis of Persian comments of Online retail stores
  faezeh forootan Mohammad Rabiei
  چكيده انگليسيThe retail market industry is one of the industries that affects the economies of countries, the life of which depends on the level of satisfaction and trust of customers to buy from these markets. In such a situation, the retail market industry is trying t More

  چكيده انگليسيThe retail market industry is one of the industries that affects the economies of countries, the life of which depends on the level of satisfaction and trust of customers to buy from these markets. In such a situation, the retail market industry is trying to provide conditions for customer feedback and interaction with retailers based on web pages and online platforms. Because the analysis of published opinions play a role not only in determining customer satisfaction but also in improving products. Therefore, in recent years, sentiment analysis techniques in order to analyze and summarize opinions, has been considered by researchers in various fields, especially the retail market industry. Manuscript profile