深度学习中numpy的一些基本操作

sigmoid 求导:

import numpy as np

def sigmoid_derivative(x):
    s = 1 / (1+np.exp(-x))
    ds = s*(1-s)
    return ds

改变arrays的形状:

两个常用的函数： np.shape() 、 np.reshape()

def image2vector(image):
    v = image.reshape(image.shape[0]*image.shape[1]*image.shape[2], 1)
    return v

normalizing rows:

It often leads to a better performance because gradient descent coverges faster after normalizations.
Here, by normalizing means changing x to \(\frac{x}{\Vert x \Vert}\)(deviding each of row vector of x by its norm.

def normalizeRows(x):
    x_norm = np.linalg.norm(x, axis=1, keepdims=True)
    x = x/x_norm
    return x

x = np.array([[0,3,4],[1,6,4]])
print("normalizeRows(x) = " + str(normalizeRows(x)))

normalizeRows(x) = [[0.         0.6        0.8       ]
 [0.13736056 0.82416338 0.54944226]]

broadcasting and softmax function:

Broadcasting is very useful for performing mathematical operations between arrays of different shapes.

softmax function: It can be thought as normalizing function used when your algorithm needs to classify two or more classes.

for \(x\in\mathbb{R}^{1\times n}\),

$softmax(x) = softmax([x_1 x_2 ... x_n])$ $ = \lbrack\frac{e^x_1}{\sum _{j}e^x_j} \frac{e^x_2}{\sum _{j}e^x_j} ... \frac{e^x_n}{\sum _{j}e^x_j}\rbrack $

for a matrix \(x\in\mathbb{R}^{m\times n}\)

$softmax(x)=\begin{pmatrix} softmax(first\ row\ of\ x)\\softmax(second\ of\ row\ of\ x) \\ ... \\softmax(last\ row\ of\ x) \end{pmatrix}$

def softmax(x):
    x_exp = np.exp(x)
    x_sum = np.sum(x_exp, axis=1, keepdims = True)
    s = x_exp / x_sum
    
    return s

L1 and L2 loss function:

L1 loss is defined as:

$L_1(\hat{y} ,y) = \sum_{i=0}^{m}|y^{(i)} - \hat{y}^{(i)}|$

def L1(yhat, y):
    
    loss = sum(abs(y-yhat))
    
    return loss

L2 loss is defined as:

$L_2(\hat{y}, y)=\sum_{i=0}^{m}(y^{(i)} - \hat{y}^{(i)})^2$

def L2(yhat, y):
    
    loss = np.dot(y-yhat, y-yhat)
    
    return loss

yhat = np.array([0.9, 0.2 ,0.1, 0.4, .9])
y = np.array([1, 0, 0, 1, 1])
print("L2 = " + str(L2(yhat, y)))

L2 = 0.43