梧席
Prerequisites for Deep Learning - Data Manipulation
Q7nl1s admin
  2023-09-07 19:37:03 2023-09-07 19:37   2024-08-08 22:58:39      1.8k Words  

EN

Textbook link: https://zh-v2.d2l.ai/chapter_preliminaries/ndarray.html

Data Manipulation

There are generally two ways to handle data in deep learning, taking Pytorch as an example

  • Python’s NumPy package: only supports CPU computation
  • Pytorch’s Tensor: supports GPU accelerated computation

Note, the data we are dealing with, i.e., n-dimensional arrays, are also called tensors.

Getting Started

Let’s first enter the jupyter environment

1
jupyter notebook

image-20230907181537323

Then find the corresponding chapter’s ipynb file, here it’s data manipulation

image-20230907181908380

Next, we can debug after importing the torch package

image-20230907182900643

image-20230907182948098

Of course, you can also modify the code for debugging

image-20230907183050688

Here’s a reminder: for better understanding, we can imagine that in a torch.zeros(...) the number of parameters determines the tensor’s dimensions. Taking the above example,

Starting from the right, the first 3 represents a vector of size 3, wrapped in one layer of brackets [].

Next, from the right, the second 4 represents a 2D matrix composed of four 1D vectors, also wrapped in one layer of brackets [].

The third 2 similarly represents a 3D tensor composed of two 2D matrices, again wrapped in one layer of brackets []. It’s quite simple.

Always remember to look from right to left.

1
2
3
4
5
6
7
8
9
10
11
torch.arange(12)             # Create a vector of 12 integers starting from 0
x.shape                      # Output the shape of x, for the above example it would be torch.Size([12])
x.numel()                    # Output the total number of elements in x, for the above example it would be 12
X = x.reshape(3, 4)          # Reshape without changing the number and values of elements, here to a 3x4 matrix, output:
                            tensor([[ 0,  1,  2,  3],
                                    [ 4,  5,  6,  7],
                                    [ 8,  9, 10, 11]])
torch.zeros((2, 3, 4))       # Initialize tensor with all zeros
torch.ones((2, 3, 4))        # Initialize tensor with all ones
torch.randn(3, 4)            # Randomly sample from a normal distribution with mean 0 and standard deviation 1 to create a (3, 4) tensor
torch.tensor([[2, 1, 4, 3], [1, 2, 3, 4], [4, 3, 2, 1]])         # Assign specific values to each element in the desired tensor by providing Python lists (or nested lists)

Operators

For any tensors with the same shape, [common standard arithmetic operators (+, -, *, /, and **) can be upgraded to element-wise operations].

Element-wise operations can be simply understood as performing point-to-point operations on specific elements of a tensor.

1
2
3
4
5
6
7
8
9
10
11
12
13
x = torch.tensor([1.0, 2, 4, 8])
y = torch.tensor([2, 2, 2, 2])
x + y, x - y, x * y, x / y, x ** y  # ** operator is exponentiation

torch.exp(x) # Exponentiation (unary operation)

X = torch.arange(12, dtype=torch.float32).reshape((3, 4))
Y = torch.tensor([[2.0, 1, 4, 3], [1, 2, 3, 4], [4, 3, 2, 1]])
torch.cat((X, Y), dim=0), torch.cat((X, Y), dim=1)    # dim=0 means concatenate along rows, dim=1 means concatenate along columns

X == Y    # Logical operator to build a binary tensor

X.sum()    # Sum of elements

Broadcasting Mechanism

Essentially, it’s about expansion.

When performing operations on two tensors of different shapes, broadcasting mechanism is triggered, as in the following case

image-20230907191757690

It essentially becomes

1
2
3
tensor([[0],           tensor([0,0],
        [1],   ->>          [1,1],
        [2]],               [2,2])

and then the transformed

1
tensor([[0, 1]])

to perform operations

To perform operations, it must satisfy: for instance, a 3x1 matrix and a 1x2 matrix can perform matrix operations.

Indexing and Slicing

This part is relatively simple and is essentially the same as Python syntax.

1
2
3
4
5
6
X[-1]           # Output the last row
X[1:3]          # Output the 4th element of the 2nd row

X[1, 2] = 9     # Change the value of the 3rd element in the 2nd row to 9

X[0:2, :] = 12  # Assign the value 12 to all elements of the 1st and 2nd rows

Saving Memory

In Python, the memory address of the result object changes after performing an operation.

When the reuse rate of this object is high and the data volume is large, we need to consider saving memory. This means assigning all values of a new object of the same shape to zero and then putting the result of the operation into this new object. The specific situation is as follows:

1
2
3
4
Z = torch.zeros_like(Y)
print('id(Z):', id(Z))
Z[:] = X + Y
print('id(Z):', id(Z))

Converting to Other Python Objects

It’s easy to [convert tensors defined by deep learning frameworks to NumPy tensors (ndarray)] and vice versa. Torch tensors and NumPy arrays will share their underlying memory, so in-place operations will change both tensors.

image-20230907193439324

CN

教材地址：https://zh-v2.d2l.ai/chapter_preliminaries/ndarray.html

数据操作

深度学习处理数据的方式大体有两种，以Pytorch为例

  • python的NumPy计算包：只支持cpu计算
  • Pytorch的Tensor：支持GPU加速计算

注意，我们要处理的数据，即n维数组，也称为张量（tensor）

入门

我们先进入jupyter环境

1
jupyter notebook

image-20230907181537323

然后找到对应章节的ipynb文件，这里是数据操作

image-20230907181908380

接着我们可以在导入torch包后进行调试

image-20230907182900643

image-20230907182948098

当然你也可以修改代码进行调试

image-20230907183050688

这里只提醒一点：为了便于理解，我们可以想象在一个 torch.zeros(...) 中有几个参数即有几维的张量，以上面为例，

从右侧看起，第一个3表示一个大小为3的向量，由一层[]包裹，

接着看到第右侧起二个4，代表四个一维向量构成的二维矩阵，外侧也由一层[]包裹，

第三个2也是如此，代表两个二维矩阵构成的三轴张量，外侧也由一层[]包裹，看是不是很简单。

牢记从右往左看的思想

1
2
3
4
5
6
7
8
9
10
11
torch.arange(12) 			#创建从0开始的12个整数的行向量
x.shape 					#输出x的形状，以上面为例则是torch.Size([12])
x.numel()					#输出x的元素总数，以上面为例则是12
X = x.reshape(3, 4)			#形状而不改变元素数量和元素值，这里是改成一个3x4的矩阵，输出为：
                            tensor([[ 0,  1,  2,  3],
                                    [ 4,  5,  6,  7],
                                    [ 8,  9, 10, 11]])
torch.zeros((2, 3, 4))		#使用全零来初始化张量
torch.ones((2, 3, 4))		#使用全一来初始化张量
torch.randn(3, 4)			#使用均值为0，标准差为1的正态分布进行随机取样，创建(3，4)的张量
torch.tensor([[2, 1, 4, 3], [1, 2, 3, 4], [4, 3, 2, 1]])			#通过提供包含数值的Python列表（或嵌套列表），来为所需张量中的每个元素赋予确定值

运算符

对于任意具有相同形状的张量， [常见的标准算术运算符（+、-、\*、/和\**）都可以被升级为按元素运算]。

按元素运算简单理解就是，可以只对一个张量的某一个特定元素进行点对点运算

1
2
3
4
5
6
7
8
9
10
11
12
13
x = torch.tensor([1.0, 2, 4, 8])
y = torch.tensor([2, 2, 2, 2])
x + y, x - y, x * y, x / y, x ** y  # **运算符是求幂运算

torch.exp(x) #求幂运算(一元运算)

X = torch.arange(12, dtype=torch.float32).reshape((3,4))
Y = torch.tensor([[2.0, 1, 4, 3], [1, 2, 3, 4], [4, 3, 2, 1]])
torch.cat((X, Y), dim=0), torch.cat((X, Y), dim=1)		#dim为0表示按行连结（自成一行），dim为1表示按列连结（自成一列）

X == Y	#逻辑运算符构建二元张量

X.sum()	#元素求和

广播机制

本质就是扩充

在两个形状不同的张量进行运算的时候，就会触发广播机制，如下面的案例

image-20230907191757690

实质一个就是

1
2
3
tensor([[0],		    tensor([0,0],
        [1],   ->>   		   [1,1],
        [2]],				   [2,2])

在和进行了变换的

1
tensor([[0, 1]])

进行运算

要进行运算需要满足：如 3x1 的矩阵和 1x2 的矩阵可以运算，即矩阵运算

索引和切片

这部分相对较简单和python语法基本一致

1
2
3
4
5
6
X[-1]			#输出倒数第一行
X[1:3]			#输出第2行的第4个元素

X[1, 2] = 9		#修改第2行第3个元素的值为9

X[0:2, :] = 12	#将第1、2行的所有元素赋值为12

节省内存

python中计算结果对象的内存地址在进行一次运算后会改变

当这个对象的复用率很高且数据量比较大时，我们就要考虑采用节省内存的方式，即将一个新的形状一样的对象全部赋值为零，再将运算的结果放到这个新的对象中，具体情况如下：

1
2
3
4
Z = torch.zeros_like(Y)
print('id(Z):', id(Z))
Z[:] = X + Y
print('id(Z):', id(Z))

转换为其他Python对象

将深度学习框架定义的张量[转换为NumPy张量（ndarray）]很容易，反之也同样容易。 torch张量和numpy数组将共享它们的底层内存，就地操作更改一个张量也会同时更改另一个张量。

image-20230907193439324

 Comments
Comment plugin failed to load
Loading comment plugin
  1. EN
    1. Data Manipulation
    2. Getting Started
    3. Operators
    4. Broadcasting Mechanism
    5. Indexing and Slicing
    6. Saving Memory
    7. Converting to Other Python Objects
  2. CN
    1. 数据操作
    2. 入门
    3. 运算符
    4. 广播机制
    5. 索引和切片
    6. 节省内存
    7. 转换为其他Python对象
© 2020 - 2024    Q7nl1s
Powered by Hexo & Theme Keep
Unique Visitor Page View
  1. EN
    1. Data Manipulation
    2. Getting Started
    3. Operators
    4. Broadcasting Mechanism
    5. Indexing and Slicing
    6. Saving Memory
    7. Converting to Other Python Objects
  2. CN
    1. 数据操作
    2. 入门
    3. 运算符
    4. 广播机制
    5. 索引和切片
    6. 节省内存
    7. 转换为其他Python对象