dl::optimization::SGD< T > Class Template Reference

Stochastic Gradient Descent optimizer with autograd support. More...

#include <optimizers.hpp>

Inheritance diagram for dl::optimization::SGD< T >:

Collaboration diagram for dl::optimization::SGD< T >:

Public Member Functions
	SGD (std::vector< Variable< T > * > parameters, T lr, T momentum=0.0, T weight_decay=0.0, bool nesterov=false)
	Constructor.
void	step () override
	Perform one SGD step.
T	get_lr () const override
	Get learning rate.
void	set_lr (T lr) override
	Set learning rate.
Public Member Functions inherited from dl::optimization::AutogradOptimizer< T >
	AutogradOptimizer (std::vector< Variable< T > * > parameters)
	Constructor.
virtual	~AutogradOptimizer ()=default
virtual void	zero_grad ()
	Zero gradients of all parameters.

Additional Inherited Members
Protected Attributes inherited from dl::optimization::AutogradOptimizer< T >
std::vector< Variable< T > * >	parameters_

Detailed Description

template<typename T>
class dl::optimization::SGD< T >

Stochastic Gradient Descent optimizer with autograd support.

Updates parameters using: param = param - lr * grad

Supports:

• Basic SGD
• Momentum
• Weight decay
• Nesterov momentum

Definition at line 79 of file optimizers.hpp.

Constructor & Destructor Documentation

SGD()

template<typename T >

dl::optimization::SGD< T >::SGD	(	std::vector< Variable< T > * >	parameters,
		T	lr,
		T	momentum = 0.0,
		T	weight_decay = 0.0,
		bool	nesterov = false
	)

Constructor.

Parameters

parameters	Parameters to optimize
lr	Learning rate
momentum	Momentum factor (default: 0)
weight_decay	Weight decay (L2 penalty) (default: 0)
nesterov	Enable Nesterov momentum (default: false)

Definition at line 9 of file optimizers.cpp.

Member Function Documentation

get_lr()

template<typename T >

T dl::optimization::SGD< T >::get_lr ( ) const

inlineoverridevirtual

Get learning rate.

Implements dl::optimization::AutogradOptimizer< T >.

Definition at line 103 of file optimizers.hpp.

set_lr()

template<typename T >

void dl::optimization::SGD< T >::set_lr ( T  lr )

inlineoverridevirtual

Set learning rate.

Implements dl::optimization::AutogradOptimizer< T >.

Definition at line 108 of file optimizers.hpp.

step()

template<typename T >

void dl::optimization::SGD< T >::step ( )

overridevirtual

Perform one SGD step.

Implements dl::optimization::AutogradOptimizer< T >.

Definition at line 31 of file optimizers.cpp.

---

The documentation for this class was generated from the following files:

• include/optimization/optimizers.hpp
• src/optimization/optimizers.cpp