Meeting Minutes from CellStrat AI Lab session on Saturday 21st Dec 2019 at Bengaluru
#CellStratAILab #disrupt4.0 #WeCreateAISuperstars
Last Saturday, we had amazing presentations by some of our AI Lab members.
First Shreyas S K presented an extensive and superb presentation on Anti-Money Laundering with help of Machine Learning. Money Laundering refers to taking advantages of banking system and perpetrating fraudulent transactions for unauthorized gains. Machine Learning algorithms like Logistic Regression, Random Forest, Gradient Boosting and XGBoost can detect laundered banking transactions from a dataset. The challenges include datasets which are unbalanced, non-representative data etc. Shreyas demonstrated a project where he demonstrated that XGBoost (Extreme Gradient Boosting) outperformed other ML algorithms for a certain banking transaction dataset which was unbalanced.
Next came an excellent overview of BERT and it’s advanced variation ALBERT by Indrajit Singh. BERT (Bidirectional Encoder Representations from Transformers) is a state-of-the-art pre-trained model on a large text corpus (Wikipedia) and it provides highly accurate solutions for NLP tasks such as Question Answering (on SQuaD dataset), Natural Language Inference, Next Sentence Prediction (NSP) etc. However BERT has huge number of parameters and hardware restrictions with GPUs/TPUs can be restricted due to memory issues. Additionally BERT can involve long training times and model degradation beyond certain capacity. The figure below shows that the model saturates on accuracy after a certain level.
ALBERT is a “lite” version of BERT and helps mitigate the BERT issues. It is basically a parameter reduction method which needs lower memory and can be faster than BERT. ALBERT led to new SOTA (state-of-the-art) benchmarks on the GLUE, RACE and SQuaD datasets. ALBERT also uses a new self-supervised loss that works on inter-sentence coherence – this helps the downstream tasks with multi-sentence inputs (with references take from https://arxiv.org/abs/1909.11942). In the charts below, we observe that the transitions from layer to layer are much smoother for ALBERT than for BERT. These results show that weight-sharing has an effect on stabilizing network parameters.
Finally, Amit Kumar presented an amazing session on Image Captioning which involves an Encoder-Decoder architecture (Encoder with CNN and Decoder with LSTM). Amit explained the basics of CNN and LSTM in great detail. Then he demonstrated an image captioning demo on the Flickr8k dataset. This dataset contains 6000 training images, 1000 validation images and 1000 testing images. Each image has 5 captions describing it – a multi-label dataset which helps our model train more accurately. A CNN encoder (we use a pre-trained VGG16 model here) accepts an image and extracts image features. For the image captioning model, the CNN dispenses with the traditional softmax classifier at the end – instead the output logits of the fully-connected layer is fed to an LSTM for producing a sequential image caption.
CellStrat AI Lab is now well-recognized across the AI industry in India as a leading AI Lab – we are chasing deeper R&D and product development and our community is growing rapidly. Join us for global AI disruption. Attend our Saturday AI Lab meetup in Bengaluru :-
BLR AI Lab meetup :-
Register : https://www.meetup.com/Disrupt-4-0/events/vcqljryzqblc/
Topic : Model Free Learning using Monte Carlo and TD, Speech Recognition
Date : Saturday 28th Dec 2019, 10:30 AM – 5 PM
Presenters : Salim Ansari, Sujith Kamath
See you this weekend for the AI Lab session ! Let’s innovate with AI !
Questions ? Call me at +91-9742800566 !
Co-Founder & Chief Data Scientist, CellStrat