HAZEL

https://arxiv.org/abs/1907.11692 RoBERTa: A Robustly Optimized BERT Pretraining Approach Language model pretraining has led to significant performance gains but careful comparison between different approaches is challenging. Training is computationally expensive, often done on private datasets of different sizes, and, as we will show, hyperpar arxiv.org 논문 발표 PPT

NLP 논문 스터디에서 발표한 내용으로, PPT만 있는 글 입니다. - 추후에 설명 글도 첨가할 예정 ** arxiv.org/abs/1909.11942 ALBERT: A Lite BERT for Self-supervised Learning of Language Representations Increasing model size when pretraining natural language representations often results in improved performance on downstream tasks. However, at some point further model increases become harder due to GPU/TPU memory limitations and longer..

NLP 논문 스터디에서 발표한 내용으로, PPT만 있는 글 입니다. - 추후에 설명 글도 첨가할 예정 ** arxiv.org/abs/1901.02860 Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Transformers have a potential of learning longer-term dependency, but are limited by a fixed-length context in the setting of language modeling. We propose a novel neural architecture Transformer-XL that enables learning dependency beyond a ..

NLP 논문 스터디에서 발표한 내용으로, PPT만 있는 글 입니다. - 추후에 설명 글도 첨가할 예정 ** arxiv.org/abs/1706.03762 Attention Is All You Need The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new arxiv.org 논문 발표 PPT