Tensorflow 使用经验积累

本文主要记录了 Tensorflow 的使用经验，包括 Tensorflow 的基本使用、Tensorflow 的高级使用、Tensorflow 的实现原理等。

一些基础知识

Tensor 的一些实例

Specific Instances Of Tensors, each of these examples are specific instances of the more general concept of a tensor and can be categorize into two groups:

Let’s organize the above list of example tensors into two groups:

number, array, 2d-array # 数字，数组，二维数组
scalar, vector, matrix # 标量，向量，矩阵

The first group of three terms (number, array, 2 d-array) are terms that are typically used in computer science, while the second group (scalar, vector, matrix) are terms that are typically used in mathematics.

Tensor 的乘法

$$
output[\cdots,i,j] = {sum-k}(a[\cdots,i,k]*b[\cdots, k,j])
$$

$$
\begin{matrix} &\begin{matrix}a:2 \times 3 \times 2 \ b: 2 \times 2 \times 3\end{matrix} & \Rightarrow \begin{matrix} ab:2\times3\times3 \ ba:2\times 2 \times 2 \end{matrix}\end{matrix}
$$

神经网络训练，第一个维度是 batch，是因为第一个维度不参与求和过程（和神经网络框架有关， TensorFlow 是这样的）。

设置 Batch Size

Keras Lambda Layer

遇到的问题及解决方案

• W tensorflow/core/common_runtime/bfc_allocator. Cc: 360] Garbage collection: deallocate free memory regions (i.e., allocations) so that we can re-allocate a larger region to avoid OOM due to memory fragmentation. If you see this message frequently, you are running near the threshold of the available device memory and re-allocation may incur great performance overhead. You may try smaller batch sizes to observe the performance impact. Set TF_ENABLE_GPU_GARBAGE_COLLECTION=false if you’d like to disable this feature.

  Solution: define environment variables

  Tensorflow issues & solution

• No gradients provided for any variable

  TensorFlow: ‘ValueError: No gradients provided for any variable’

• X and y must have the same dtype, got tf. Float 32 != tf. Int 32

  Tensorflow error(二):x and y must have the same dtype, got tf.float32 != tf.int32_源的博客-CSDN 博客

• Cannot convert an unknown Dimension to a Tensor: ?

  将 tensor.shape[0] 改成 tf.shape(tensor)[0]

  Tensorflow Debug 指南

• 将 float 64 张量转换为 float 32

  TensorFlow：将 float64 张量转换为 float32 - 问答 - Python 中文网

• 关于 loss 的写法

  写 loss function 的时候，需要使用 tensor 来作比较，这样生成的梯度才能和参数有关，才可以训练。如果把 tensor 转为 numpy，则 loss 和参数无关。

TensorFlow Breaking Changes

• tf2.10 是最后支持 Windows 原生 GPU 的版本

  Announcing TensorFlow Official Build Collaborators — The TensorFlow Blog

  2.10 last version to support native Windows GPU - General Discussion - TensorFlow Forum

• tf2.15 是最后默认安装 keras2 的版本，tf2.16 以后的版本会默认安装 keras3

  Keras Newsletter (December 1, 2023) - Keras - TensorFlow Forum