Multi-head attention

CellStrat > Research/Blog > Multi-head attention
19
May
Transformers for NLP

#CellStratAILab #disrupt4.0 #WeCreateAISuperstars #WhereLearningNeverStops Last Saturday, our AI Lab Researcher Indrajit Singh presented an exhaustive webinar on Transformers, which are used in NLP. Introduction To combine the advantages from both CNNs and RNNs, [Vaswani et al., 2017] designed a novel architecture using the attention mechanism. This architecture, which is called as Transformer, achieves parallelization by […]

Posted in: Natural Language Processing,
Tags: AddNorm, deep learning, encoder-decoder, Multi-head attention, MXNet, natural language processing, NLP, position-wise FFN, positional encoding, Residual block, self-attention, seq2seq, Sequence-to-sequence, skip connection, tensorflow, transformer, transformerdecoder, transformerencoder,
07
May
Sequence to Sequence Learning and Attention in Neural Networks

#CellStratAILab #disrupt4.0 #WeCreateAISuperstars #AlwaysUpskilling Last Saturday 2 May 2020, our AI Lab Researcher Indrajit Singh presented a fabulous session on “Sequence to Sequence Learning and Attention in Neural Networks“. Sequence to Sequence Model :- Sequence-to-sequence learning (Seq2Seq) is about training models to convert sequences from one domain (e.g. sentences in English) to sequences in another […]

Posted in: Natural Language Processing,
Tags: Attention is All You Need, attention model, Attentional Interfaces, encoder-decoder, Long Short-term memory models, LSTM, machine translation, Multi-head attention, NLP, NLP natural language processing, Recurrent Neural Network, RNN, S2S Model, Scaled Dot-Product Attention, Sequence-to-sequence,