A comprehensive C++ library for implementing and learning both deep learning and traditional machine learning algorithms from scratch, featuring modern C++ design patterns, extensive documentation, and automated CI/CD.

[](LICENSE)

🎯 Project Goals

This project provides a structured framework for implementing fundamental deep learning and traditional machine learning algorithms in C++. It's designed for educational purposes and hands-on learning of:

Deep Learning Algorithms:

Neural network architectures (Feedforward, CNN, RNN, LSTM, GRU)
Optimization algorithms (SGD, Adam, RMSprop)
Activation functions (ReLU, Sigmoid, Tanh, Softmax, LeakyReLU)
Loss functions (MSE, Cross-entropy, Hinge loss)

Traditional Machine Learning Algorithms:

Dimensionality reduction (Principal Component Analysis)
Clustering algorithms (K-Means)
Classification algorithms (Support Vector Machine)

Utilities:

Mathematical utilities and high-performance matrix operations
Data processing with comprehensive loading and preprocessing utilities

🚀 Key Features

📚 Comprehensive Documentation: Full Doxygen documentation with examples and mathematical descriptions
🔧 Modern C++: Uses C++23 features and best practices
🧪 Tested: Comprehensive test suite with Google Test
🔄 CI/CD: Automated testing, static analysis, and documentation deployment
📊 Performance: Optimized matrix operations and memory management
🎓 Educational: Detailed comments and learning-focused design

📁 Project Structure

deep-learning-algo-impls/
├── include/                    # Header files
│   ├── neural_networks/        # Deep learning architectures
│   │   ├── feedforward.hpp     # Feedforward neural networks
│   │   ├── cnn.hpp            # Convolutional neural networks
│   │   └── rnn.hpp            # Recurrent neural networks (RNN/LSTM/GRU)
│   ├── optimization/           # Optimization algorithms
│   │   └── optimizers.hpp     # SGD, Adam, RMSprop optimizers
│   ├── activation/             # Activation functions
│   │   └── functions.hpp      # ReLU, Sigmoid, Tanh, Softmax
│   ├── loss/                   # Loss functions
│   │   └── functions.hpp      # MSE, Cross-entropy, Hinge loss
│   ├── ml/                     # Traditional ML algorithms
│   │   ├── ml.hpp             # Main ML header (includes all algorithms)
│   │   ├── pca.hpp            # Principal Component Analysis
│   │   ├── kmeans.hpp         # K-Means clustering
│   │   └── svm.hpp            # Support Vector Machine
│   └── utils/                  # Utility classes
│       ├── matrix.hpp         # Matrix operations
│       └── data_loader.hpp    # Data loading and preprocessing
├── src/                        # Implementation files
│   ├── neural_networks/        # Deep learning implementations
│   ├── optimization/           # Optimizer implementations
│   ├── activation/             # Activation function implementations
│   ├── loss/                   # Loss function implementations
│   ├── ml/                     # Traditional ML implementations
│   │   ├── pca.cpp            # PCA implementation
│   │   ├── kmeans.cpp         # K-Means implementation
│   │   └── svm.cpp            # SVM implementation
│   └── utils/                  # Utility implementations
├── tests/                      # Unit tests
│   ├── test_feedforward.cpp    # Neural network tests
│   ├── test_matrix.cpp        # Matrix operation tests
│   └── test_optimizers.cpp    # Optimizer tests
├── .github/workflows/          # CI/CD pipelines
│   └── ci.yml                 # Automated testing workflow
├── CMakeLists.txt             # Build configuration
├── Doxyfile                   # Documentation configuration
└── main.cpp                   # Example usage

📖 Documentation

Full API documentation is automatically generated using Doxygen with the modern Doxygen Awesome theme and deployed to GitHub Pages:

🔗 View Documentation

The documentation features:

Modern, clean design with improved readability <mcreference link="https://jothepro.github.io/doxygen-awesome-css/" index="1">1</mcreference>
Mobile-responsive interface for documentation on any device <mcreference link="https://jothepro.github.io/doxygen-awesome-css/" index="1">1</mcreference>
Dark mode support for comfortable viewing <mcreference link="https://jothepro.github.io/doxygen-awesome-css/" index="1">1</mcreference>
Enhanced navigation with sidebar treeview
Complete API reference with examples
Mathematical descriptions of algorithms
Usage patterns and best practices
Implementation guides and tutorials

🚀 Quick Start

Matrix Operations

#include "utils/matrix.hpp"
using namespace dl::utils;
 
// Create matrices
Matrix<double> a(3, 3, 1.0);  // 3x3 matrix filled with 1.0
Matrix<double> b = Matrix<double>::random(3, 3);  // Random 3x3 matrix
 
// Matrix operations
auto c = a * b;  // Matrix multiplication
auto d = a + b;  // Element-wise addition
auto e = a.transpose();  // Transpose

Principal Component Analysis

#include "utils/pca.hpp"
#include "utils/matrix.hpp"
using namespace dl::utils;
 
// Create a dataset
MatrixD data({
    {2.5, 2.4},
    {0.5, 0.7},
    {2.2, 2.9},
    {1.9, 2.2},
    {3.1, 3.0}
});
 
// Create a PCA object
PCAD pca;
 
// Fit the PCA model to the data
pca.fit(data);
 
// Get the explained variance ratio
auto variance_ratio = pca.explained_variance_ratio();
for (size_t i = 0; i < variance_ratio.size(); ++i) {
    std::cout << "Component " << i << ": " << variance_ratio[i] << std::endl;
}
 
// Reduce dimensions (e.g., to 1D)
MatrixD reduced_data = pca.transform(data, 1);

Neural Network Training

#include "neural_networks/feedforward.hpp"
#include "utils/data_loader.hpp"
using namespace dl;
 
// Define network architecture
std::vector<size_t> layers = {784, 128, 64, 10};  // MNIST-like network
neural_networks::FeedforwardNetwork network(layers);
 
// Load and preprocess data
auto [features, labels] = utils::CSVLoader::load_features_labels(
    "data.csv", {0, 1, 2, 3}, {4});
utils::Dataset<double> dataset(features, labels);
 
// Train the network
network.train(dataset, epochs=100, learning_rate=0.01);
 
// Make predictions
auto predictions = network.predict(test_features);

Data Loading and Preprocessing

#include "utils/data_loader.hpp"
using namespace dl::utils;
 
// Load CSV data
auto data = CSVLoader::load_csv("dataset.csv");
 
// Preprocess data
auto normalized = Preprocessor::normalize(data, 0.0, 1.0);
auto standardized = Preprocessor::standardize(data);
 
// Split dataset
auto [train, val, test] = Preprocessor::train_val_test_split(
    dataset, 0.7, 0.15);
 
// Create data loader for batch processing
DataLoader<double> loader(train, batch_size=32, shuffle=true);
while (loader.has_next()) {
    auto [batch_features, batch_labels] = loader.next_batch();
    // Process batch...
}

🛠️ Prerequisites

C++23 compatible compiler (GCC 11+, Clang 14+, or MSVC 2022+)
CMake 3.31 or higher
Google Test for unit testing
Git for version control

Installing Dependencies

Ubuntu/Debian

sudo apt-get update
sudo apt-get install -y cmake ninja-build libgtest-dev
sudo apt-get install -y gcc-11 g++-11  # or clang-14

macOS

brew install cmake ninja googletest

Windows (vcpkg)

vcpkg install gtest

🚀 Building the Project

Clone the repository
git clone <repository-url>

cd deep-learning-algo-impls
Configure and build
cmake -B build -DCMAKE_BUILD_TYPE=Release

cmake --build build
Run tests
cd build

ctest --output-on-failure
Run the main executable
./build/deep_learning_algo_impls

📚 Implementation Guide

This project provides header files with comprehensive TODO comments and example structures. Each algorithm should be implemented following these guidelines:

1. Neural Networks

Feedforward Networks: Implement basic multilayer perceptrons with configurable architectures
CNNs: Add convolution, pooling, and feature extraction layers
RNNs: Implement sequence processing with LSTM and GRU variants

2. Optimization

SGD: Basic gradient descent with momentum support
Adam: Adaptive learning rates with bias correction
RMSprop: Root mean square propagation

3. Mathematical Utilities

Matrix Class: Efficient matrix operations for linear algebra
Activation Functions: Differentiable activation functions
Loss Functions: Various loss functions for different tasks

4. Data Processing

Data Loaders: CSV and image data loading utilities
Preprocessing: Normalization, standardization, and augmentation

🧪 Testing Strategy

The project includes comprehensive unit tests for:

Matrix operations and mathematical correctness
Neural network forward/backward propagation
Optimizer convergence and update rules
Activation and loss function derivatives

Running Specific Tests

# Run all tests
ctest
 
# Run specific test suite
./build/run_tests --gtest_filter="MatrixTest.*"
 
# Run with verbose output
./build/run_tests --gtest_filter="*" --gtest_output="verbose"

🔄 Continuous Integration

The project includes GitHub Actions workflows that automatically:

Build and test on multiple platforms (Ubuntu, macOS)
Test with different compilers (GCC, Clang)
Run static analysis and code formatting checks
Generate documentation (when implemented)
Perform memory leak detection

📖 Learning Path

Recommended implementation order for learning:

Start with Matrix utilities - Foundation for all operations
Implement activation functions - Simple mathematical functions
Build feedforward networks - Core neural network concepts
Add optimization algorithms - Learning and convergence
Implement loss functions - Training objectives
Extend to CNNs - Spatial data processing
Add RNNs/LSTMs - Sequential data processing

🤝 Contributing

This is a learning-focused project. Feel free to:

Implement the TODO items in the headers
Add comprehensive tests for your implementations
Improve documentation and examples
Optimize performance and memory usage
Add new algorithms and techniques

📄 License

Apache License 2.0

🔗 Resources

Deep Learning Book by Ian Goodfellow

Happy Learning! 🚀