import os
! wget https://www.python.org/static/img/python-logo.png
import PIL
img = PIL.Image.open ("python-logo.png")
img
Popular frameworks:
* Caffe, Caffe2, CNTK, Kaldi, DL4J, Keras – a set of modules for design;
* TensorFlow, Theano, MXNet – graph programming;
* Torch and PyTorch – register the main parameters, and the graph will be built automatically.
Consider the PyTorch library (NumPy + CUDA + Autograd) because of its simplicity. Let's look at operations with tensors – multidimensional arrays. Let's connect the library and declare two tensors: press + Code, enter the code into the cell and press execute:
import torch
a = torch.FloatTensor ([[1, 2, 3], [5, 6, 7], [8, 9, 10]])
b = torch.FloatTensor ([[– 1, -2, -3], [-10, -20, -30], [-100, -200, -300]])
Element-wise operations such as "+", "-", "*", "/" on two matrices of the same dimensions perform operations with their corresponding elements:
a + b
tensor ([[0., 0., 0.],
[-5., -14., -23.],
[-92., -191., -290.]])
Another option for the elementwise operation is to apply one operation to all elements one by one, for example, multiply by -1 or apply a function:
a
tensor ([[1., 2., 3.],
[5., 6., 7.],
[8., 9., 10.]])
a * -1
tensor ([[-1., -2., -3.],
[-5., -6., -7.],
[-8., -9., -10.]])
a.abs ()
tensor ([[1., 2., 3.],
[5., 6., 7.],
[8., 9., 10.]])
There are also convolution operations, such as sum, min, max, which, as input, give the sum of all elements, the smallest or largest element of the matrix:
a.sum ()
tensor (51.)
a.min ()
tensor (1.)
a.max ()
tensor (10.)
But, we will be more interested in post-column operations (the operation will be performed on each column):
a.sum (0)
tensor ([14., 17., 20.])
a.min (0)
torch.return_types.min (values = tensor ([1., 2., 3.]), indices = tensor ([0, 0, 0]))
a.max (0)
torch.return_types.max (values = tensor ([8., 9., 10.]), indices = tensor ([2, 2, 2]))
As we remember, a neural network consists of three layers, a layer of neurons, and a neuron contains connections at the input with weights in the form of prime numbers. The weight is set by an ordinary number, then the incoming connections to the neuron can be described by a sequence of numbers – a vector (one-dimensional array or list), the length of which is the number of connections. Since the network is fully connected, all the neurons of this layer are connected to the previous one, and therefore the vectors demonstrating them also have the same length, creating a list of vectors of equal length – a matrix. It is a convenient and compact layer representation optimized for use on a computer. At the output of the neuron, there is an activation function (sigmoid or, ReLU for deep and ultra-deep networks), which determines whether the neuron outputs a value or not. To do this, it is necessary to apply it to each neuron, that is, to each column: we have already seen the operation on columns.