Posts | anurajmohan.in

NLP → Transformers: Prerequisite Learning Path

Sat, 07 Mar 2026 00:00:00 +0000

NLP → Transformers: Prerequisite Learning Path

Before learning Large Language Models (LLMs) such as GPT, BERT, Gemini, or Claude, it is important to understand some foundational concepts in Natural Language Processing (NLP) and Deep Learning.

This roadmap provides a quick step-by-step learning path from basic NLP concepts to Transformers , which are the backbone of modern LLMs.

1. Introduction to Natural Language Processing

Understand what Natural Language Processing (NLP) is and why it is important in AI.

Topics to Learn

What is NLP
Applications of NLP
Text preprocessing
Tokenization
Stopword removal
Stemming and Lemmatization

Learning Resource

Natural Language Processing In 5 Minutes | What Is NLP And How Does It Work?

2. Word Representations

Machines cannot understand raw text directly.
Words must be converted into numerical vectors.

Topics to Learn

Bag of Words
TF-IDF
Word Embeddings

Learning Resource

What are Word Embeddings?

3. Neural Networks for NLP

Deep learning models are widely used for NLP tasks.

Topics to Learn

Basics of Neural Networks
Forward propagation
Backpropagation
Activation functions
Neural networks for text processing

Learning Resource

Neural Network In 5 Minutes | What Is A Neural Network?

4. Sequence Models

Text is sequential in nature, so models must understand context and order of words.

Topics to Learn

Recurrent Neural Networks (RNN)
Long Short-Term Memory (LSTM)
Gated Recurrent Units (GRU)
Sequence modeling

Learning Resource

[Illustrated Guide to Recurrent Neural Networks] (https://youtu.be/LHXXI4-IEns?si=IiUFY9F5uMYyCptP)

5. Transformers

Transformers are the foundation of modern LLMs.

They overcome the limitations of RNNs and allow models to process sequences in parallel using attention mechanisms.

Topics to Learn

Attention mechanism
Self-attention
Transformer architecture
Encoder–decoder structure
Positional encoding

Learning Resource

What are Transformers (Machine Learning Model)?

Next Step: Large Language Models

Once you understand Transformers, you can move to:

BERT
GPT models
Generative AI
Retrieval Augmented Generation (RAG)
Agentic AI

These concepts are build on the Transformer architecture.

A Self-Learning Roadmap to Deep Learning

Wed, 18 Feb 2026 00:00:00 +0000

This roadmap is designed for undergraduate students transitioning from Machine Learning to Deep Learning. It covers Neural Networks, CNNs, RNNs, LSTMs, and Transformers using PyTorch.

🔵 Phase 1: Deep Learning Foundations

1️⃣ Neural Network Fundamentals

✅ Core Concepts

What is a Neural Network?
Perceptron
Activation Functions (ReLU, Sigmoid, Tanh, Softmax)
Forward Propagation
Loss Functions (MSE, Cross-Entropy)
Backpropagation
Gradient Descent
Overfitting & Regularization (Dropout, L2)

2️⃣ Math Behind Deep Learning (Revision)

📌 Linear Algebra

Matrix multiplication
Dot product
Vector spaces
Eigenvalues (basic intuition)

📌 Calculus

Partial derivatives
Chain rule
Gradient computation

📌 Probability

Softmax as probability distribution
Log-likelihood
Cross-entropy loss

📚 Learning Resources (Foundations)

🎥 YouTube Playlists

🎓 Courses

3️⃣ PyTorch Basics

Students must learn:

Working with tensors
Autograd (automatic differentiation)
Implementing a three-layer neural network
Writing a full training loop
Model evaluation
GPU training

📚 Resources

🔵 Phase 2: Core Deep Learning Architectures

📘 Feed-Forward Neural Networks (FNN / MLP)

Topics

Multi-layer neural networks
Activation functions
Loss functions
Backpropagation
Weight initialization

Recommended Resources

📘 Convolutional Neural Networks (CNN)

Topics

Convolution operation
Filters and feature maps
Stride and padding
Pooling layers
CNN architectures
Training CNN in PyTorch

Recommended Resources

📘 Recurrent Neural Networks (RNN, LSTM, GRU)

Topics

Sequential data
Vanishing gradient problem
Basic RNN
LSTM
GRU
Next-word prediction

Recommended Resources

📘 Transformers

Topics

Attention mechanism
Self-attention
Scaled dot-product attention
Multi-head attention
Positional encoding
Encoder–Decoder architecture
Fine-tuning pretrained models

Recommended Resources

🔵 Phase 3: Mini Projects (Choose Any 2)

🔹 Option A: Image Classification (CNN)

Dataset:
CIFAR-10 Dataset

Deliverables

Data preprocessing
CNN architecture
Training curves
Accuracy
Confusion matrix
Conclusion

🔹 Option B: Sentiment Analysis (RNN / LSTM)

Dataset:
IMDB Reviews Dataset

Deliverables

Text preprocessing
Tokenization
LSTM model
F1-score evaluation
Error analysis

🔹 Option C: Text Summarization (Transformer)

Dataset:
CNN/DailyMail Dataset

Deliverables

Fine-tuning pretrained model
ROUGE evaluation
Generated samples

🎯 Expected Outcomes

After completing this roadmap, students will be able to:

Build neural networks from scratch
Train CNN, RNN, and LSTM models
Understand and implement Transformers
Fine-tune pretrained models
Write structured DL project reports
Apply Deep Learning in internships and projects

A Self-Learning Roadmap to Machine Learning

Wed, 18 Feb 2026 00:00:00 +0000

📌 Overview

This roadmap is designed for students starting Machine Learning from scratch.
It is split into two phases:

Phase 1: Foundations (Math + Python)
Phase 2: ML Concepts + Practice + Mini Projects

Phase 1: Machine Learning Foundations

1) Math for Machine Learning

Before starting ML, students should be comfortable with the following topics.

✅ Topics to Learn

Linear Algebra

Vectors, matrices
Eigenvalues, eigenvectors

Calculus & Gradients

Derivatives
Chain rule
Gradients

Probability & Statistics

Distributions
Expectation
Mean, variance

⭐ Best Resources

Linear Algebra

3Blue1Brown Playlist

Calculus

3Blue1Brown Playlist

Probability and Statistics

StatQuest Playlist

🎓 Courses (Optional but Recommended)

2) Python for Data Science

Students should become confident in using Python for data handling and visualization.

✅ Skills Students Must Learn

NumPy — numerical computing
Pandas — data manipulation
Matplotlib / Seaborn — visualization
scikit-learn — ML models (after ML basics)
PyTorch — deep learning (after DL basics)

⭐ Resources

Phase 2: Machine Learning Concepts + Practice

1) ML Core Concepts

Students should clearly understand:

Features (X) and Target (y)
Train vs Test split
Overfitting vs Underfitting

2) Models to Learn (Beginner Level)

Start with these 3 models:

Linear Regression
Logistic Regression
Decision Trees

Also learn the basics of:

Representation → Loss Function → Optimization

3) Evaluation Metrics

Regression Metrics

MAE
MSE
RMSE

Classification Metrics

Accuracy
Confusion Matrix
Precision, Recall, F1-score

📚 Resources for Phase 2

Theory Resources

Hands-on Resources

🧪 Mini Projects (Choose Any 2)

Students must complete any 2 projects from the list below.

Option A: House Price Prediction (Regression)

Dataset:

House Prices Kaggle Dataset

Deliverables:

Preprocessing
Model training
RMSE evaluation
Conclusion

Option B: Titanic Survival Prediction (Classification)

Dataset:

Titanic Kaggle Dataset

Deliverables:

Encoding + preprocessing
Model training
Confusion matrix
Precision/Recall

Option C: Student Performance Prediction

Dataset:

Students Performance Dataset

Deliverables:

EDA
Correlation analysis
Model training
Evaluation

Option D: Diabetes Prediction

Dataset:

Pima Indians Diabetes Dataset

Deliverables:

Classification model
F1-score evaluation
Conclusion

📦 Final Submission Format (Phase 1 + Phase 2)

Students must submit one notebook/report containing:

Dataset loading
Data cleaning + missing value handling
Exploratory Data Analysis (EDA) + plots
Feature engineering (basic)
Model training
Evaluation metrics
Final conclusion (5–10 lines)

✅ Outcome

By the end of Phase 1 and Phase 2, students will be able to:

Understand ML fundamentals clearly
Build beginner ML models using scikit-learn
Evaluate models properly
Complete 2 end-to-end mini projects
Write a clean ML notebook/report for submission

Introduction to Geometric Deep Learning

Tue, 01 Feb 2022 00:00:00 +0000

A Collection of good reads on Geometric Deep Learning

👉 https://geometricdeeplearning.com/
👉 Blog on Medium
👉 Paper on Arxiv
👉 Python based Tool - Pytorch Geometric

Introduction to Network Representation Learning

Sat, 19 Jun 2021 00:00:00 +0000

Get familarized with Network Representation Learning

👉 Stanford Link
👉 Blog on towardsdatascience
👉 E-book
👉 Must Read Papers - Collection in Github
👉 Google AI blog
👉 Python based Tool on Graph Neural Network - DGL
👉 Python based Tool for Graph Machine Learning - StellarGraph

Concepts and Tools for Social Network Analysis & Mining

Fri, 09 Apr 2021 00:00:00 +0000

Concepts and Tools for Social Network Analysis & Mining

👉 Papers on Social Network Analysis
👉 Blog on towardsdatascience
👉 E-book
👉 Google AI Research Page on Graph Mining
👉 Stanford Network Datasets
👉 Python based Tool for Network Analysis - NetworkX