Datum održavanja: 20. srpnja 2021. u 10:00 sati u učionici S-032 uz online prijenos preko BBB platforme
Predavač: Karlo Babić, Odjel za informatiku, Sveučilište u Rijeci
Naziv predavanja: Omnidirectional Recursive training

Abstract:

The omnidirectional recursive training method is a method for training a neural network for text representation by constructing a pyramidal hierarchy for text input. Each higher level of this constructed pyramid has higher-level representations of the input text abstracting it from letters or subwords to words, phrases, sentences, and eventually perhaps paragraphs and entire documents.

At each level of the pyramid, the method encodes each successive pair of representations into a higher-level representation. This representation is then used for two tasks: decoding the representation into the pair of representations it was encoded with (encoder-decoder module), and predicting the representations to the left and right of the encoded pair of representations (regression module).

The encoder-decoder module can be used to decode the entire pyramid from the top-level representation to the bottom-level representations.

The regression module is used to predict the neighboring letters, subwords, words, etc., and learn a representation space in which the relationship between the text representations is meaningful, making the text representations useful for tasks such as text similarity and other tasks that require representing text as a vector.