Programmer Think

2021SC@SDUSC 0. Preface This blog post is mainly about the self attention mechanism of deep learning, qkv part and gradient in deep learning 1. Self attention mechanism of deep learning 1.1 basic concepts of self attention mechanism For the self attention mechanism, Q, K and V, that is, query, key and value, all come from the same inpu ...

Preface: I haven't sorted it out for a long time, but I'm actually lazy and haven't learned much, so I'll briefly talk about what I've learned at the beginning, and talk about the problems encountered in learning and a little trivial experience. (the last part of the previous article is about the tips of variable types) Small review: 1.const ...

Similar to other data preprocessing of machine learning, natural language processing also needs text preprocessing, such as word segmentation in Chinese and part of speech restoration in English. Common pretreatment methods 1. Text normalization Capital to lowercase output_str = input_str.lower() Digital processing Remove all numbers imp ...

summary The Minimum Edit Distance (MED) was proposed by Vladimir Levenshtein, a Russian scientist, in 1965, hence the name Levenshtein Distance. Levenshtein Distance is an index used to measure the similarity of two sequences in the fields of information theory, linguistics and computer science. Generally speaking, editing distance refers t ...

1 fine tune the transformer model to solve the translation task ! pip install datasets transformers "sacrebleu>=1.4.12,<2.0.0" sentencepiece model_checkpoint = "Helsinki-NLP/opus-mt-en-ro" # Select a model checkpoint As long as the pre trained transformer model contains the head layer of seq2seq structure, this notebook can theo ...

1. Introduction to Twitter Text Emotional Classification Contest Practice match address: https://www.heywhale.com/home/activity/detail/611cbe90ba12a0001753d1e9/content This exercise has 13 emotional categories, so the score is low... Twitter tweets have many features. First, unlike Facebook, tweets are text-based and can be registered and do ...

AttributeError: Can't get attribute 'xxx' on < module 'main' from 'xxx' another possible solution Problem Restatement: when learning pytorch, I learned to serialize the Word2Seq class using the pickle module and save the serialization in the ws.pkl file. Then I created the lib.py file again. This problem occurs when I deserialize ws.pkl wit ...

1. The theoretical explanation is clear and easy to understand: Understand the basic principle of "generating countermeasure network GAN" + 10 typical algorithms + 13 applications (easyai.tech) 2. Code implementation set: GitHub - eriklindernoren/Keras-GAN: Keras implementations of Generative Adversarial Networks. 3. Here is a bri ...

summary From today on, we will start a journey of natural language processing (NLP). NLP can let us process, understand and use human language to realize the communication bridge between machine language and human language key word Keywords, i.e. key words, can describe the essence of an article and have important applications in docum ...