List of Articles sequence-to-sequence models

• Open Access Article
  • Abstract Page
  • Full-Text
  
  1 - On the Behavior of Pre-trained Word Embedding Variants in Deep Headline Generation from Persian Texts
  Mohammad Ebrahim Shenassa Behrooz Minaei-Bidgoli
  Inspired by sequence-to-sequence models for machine translation, deep-learning based summarization methods were presented. The summaries generated this way, are structurally more readable and usually convey the complete meaning to the reader. In these methods, embeddi More

  Inspired by sequence-to-sequence models for machine translation, deep-learning based summarization methods were presented. The summaries generated this way, are structurally more readable and usually convey the complete meaning to the reader. In these methods, embedding vectors are used for semantic representation, in which the weight of each word vector is learned according to its neighboring words from a large corpus. In static word embedding, the weight of the vectors is obtained by choosing a proximity window for each word. But in contextual ones like BERT, multilayer transformers are applied to calculate the weight of these vectors, which pay attention to all the words in the text. So far, several papers have shown that contextual word embedding are more successful than the other ones due to the ability of fine-tuning the weights to perform a specific natural language processing task. However, the performance of the initial weights of these vectors is not investigated for headline generation from Persian texts. In this paper, we will investigate the behavior of pre-trained word embedding variants without fine-tuning in deep headline generation from Persian texts. To train the headline generation model, "Elam Net" is used, which is a Persian corpus containing about 350 thousand pairs of abstracts and titles of scientific papers. The results show that the use of BERT model, even without fine-tuning its weights, is effective in improving the quality of generated Persian headlines, bringing the ROUGE-1 metric to 42%, which is better than the other pre-trained ones. Manuscript profile