pylearn.neural_network package

Submodules

pylearn.neural_network.activation module

class pylearn.neural_network.activation.Activation(function: classmethod, derivative: classmethod)

Bases: Layer

Defines the activation functions for a layer. Inherits from Layer to compute forward pass and backpropagation with the activation function.

Attributes:

function (function):

Activation function

derivative (function):

Derivative of the activation function

backpropagation(output_gradient: ndarray, learning_rate: int) → ndarray

Derived output for the predecessor after activation.

Parameters:

output_gradient (numpy.ndarray):

input of the next layer (output = ∂E/∂Y)

Returns:

Derivative of the function as array

forward_pass(x: ndarray) → ndarray

Takes the input (x) of the layer and outputs f(x).

Parameters:

x (numpy.ndarray):

Input vector of the previous layer

Returns:

Result as array

pylearn.neural_network.activation_functions module

class pylearn.neural_network.activation_functions.ReLU

Bases: Activation

Defines the ReLU activation function for a layer. Inherits from Activation and passes the function and derivative.

class pylearn.neural_network.activation_functions.Sigmoid

Bases: Activation

Defines the Sigmoid activation function for a layer. Inherits from Activation and passes the function and derivative.

class pylearn.neural_network.activation_functions.Softmax

Bases: Activation

Defines the Softmax activation function for a layer. Inherits from Activation and passes the function and derivative.

class pylearn.neural_network.activation_functions.Tanh

Bases: Activation

Defines the Tanh activation function for a layer. Inherits from Activation and passes the function and derivative.

pylearn.neural_network.dense_layer module

class pylearn.neural_network.dense_layer.Dense_layer(input_size: int, output_size: int)

Bases: Layer

Defines the dense layer structure (fully connected layer) to use the forward pass and backpropagation.

Attributes:

weights (numpy.ndarray):

Weight matrix of the layer

bias (numpy.array):

Bias vector of the layer

backpropagation(output_gradient: ndarray, learning_rate=0.1) → ndarray

Updates the parameters depending on the derivatives.

Parameters:

output_gradient (numpy.array):

input of the next layer (output = ∂E/∂Y)

learning_rate (numpy.array, optional):

hyperparameter to determine the step size of updates, default: 0.01

Returns:

Derivative of the function as array

forward_pass(x: ndarray) → ndarray

Receives its input (x) from the previous layer and provides an output for the next layer.

Parameters:

x (numpy.ndrray):

Input vector of the previous layer

Returns:

Result as array

pylearn.neural_network.layer module

class pylearn.neural_network.layer.Layer

Bases: object

An abstract base class for neural network layers. Defines the basic properties and methods that can be used by all derived layers.

backpropagation(output_gradient, learning_rate) → None

forward_pass(input) → None

pylearn.neural_network.loss_functions module

pylearn.neural_network.loss_functions.cross_entropy_loss(y_true: ndarray, y_pred: ndarray) → float

Calculates the Cross Entropy Loss between the estimated values and the actual value.

Parameters:

y_true (numpy.ndarray):

Correct output

y_pred (numpy.ndarray):

Predicted output

Returns:

Error as float

pylearn.neural_network.loss_functions.cross_entropy_loss_derivative(y_true: ndarray, y_pred: ndarray) → ndarray

Calculates the derivative (∂E/∂Y) of the Cross Entropy Loss between the estimated values and the actual value.

Parameters:

y_true (numpy.ndarray):

Correct output

y_pred (numpy.ndarray):

Predicted output

Returns:

Derivative of CE as array

pylearn.neural_network.loss_functions.mse(y_true: ndarray, y_pred: ndarray) → float

Calculates the Mean Squared Error between the estimated values and the actual value.

Parameters:

y_true (numpy.ndarray): Correct output y_pred (numpy.ndarray): Predicted output

Returns:

Error as float

pylearn.neural_network.loss_functions.mse_derivative(y_true: ndarray, y_pred: ndarray) → ndarray

Calculates the derivative (∂E/∂Y) of the Mean Squared Error (MSE) between the estimated values and the actual value.

Parameters:

y_true (numpy.ndarray):

Correct output

y_pred (numpy.ndarray):

Predicted output

Returns:

Derivative of MSE as array

pylearn.neural_network.network module

class pylearn.neural_network.network.NeuralNetwork

Bases: object

static fit(X: ndarray, Y: ndarray, network: list, loss_function: classmethod, loss_derivative: classmethod, epochs=1000, learning_rate=0.1, log_error=False, log_details=False, log_duration=True) → None

Trains the neural network by performing forward pass and back propagation and also computes the error for every epoch.

To log: log_details, log_error

Parameters:

X (numpy.array):

Training input

Y (numpy.array):

Training output

network (list[Layer]):

List of network layers

loss_function (classmethod):

Loss function to compute the error

loss_derivative (classmethod):

Derivative of the loss function to perform backpropagation

epochs (int, optional):

Hyperparameter, number of learning iterations, default: 1000

learning_rate (int, optional):

Hyperparameter, default: 0.01

log_error (bool, optional):

Logs the error of each iteration, default: False

log_details (bool, optional):

Logs the parameters of each layer, default: False

log_duration (bool, optional):

Logs the duration of the training, default: True

Returns:

None

static predict(X: ndarray, network: list) → ndarray

The neural network computes the output of a given input X.

Parameters:

X (numpy.array):

Testing input

network (list[Layer]):

List of network layers

Returns:

The predicted output as an array

Module contents