TensorFlow Tensor Operations

Tensors are the core data structure in TensorFlow, which can be understood as an extension of the concept of multidimensional arrays.

In machine learning, almost all data is ultimately represented in tensor form for processing.

Basic Characteristics of Tensors

  1. Data type (dtype): Each tensor has a specific data type, such as tf.float32, tf.int64, etc.
  2. Shape: Represents the size of each dimension of the tensor, e.g., (2,3) represents a matrix with 2 rows and 3 columns.
  3. Device location: Indicates whether the tensor is stored on CPU or GPU.

Dimensions of Tensors

  • 0-dimensional tensor: scalar, e.g.,tf.constant(5)
  • 1-dimensional tensor: vector, e.g.,tf.constant([1,2,3])
  • 2-dimensional tensor: matrix, e.g.,tf.constant([[1,2],[3,4]])
  • 3-dimensional and above: higher-order tensors, e.g.,tf.ones((2,3,4))represents two 3×4 matrices

Common Methods for Creating Tensors

1. Creating from Python Lists/NumPy Arrays

Example

import tensorflow as tf
import numpy as np

# Create from a Python list
tensor_from_list = tf.constant([[1, 2], [3, 4]])

# Create from a NumPy array
numpy_array = np.array([[5, 6], [7, 8]])
tensor_from_numpy = tf.constant(numpy_array)

2. Creating Special Value Tensors

Example

# All-zeros tensor
zeros = tf.zeros((2, 3))  # All-zeros matrix with 2 rows and 3 columns

# All-ones tensor
ones = tf.ones((3, 2))    # All-ones matrix with 3 rows and 2 columns

# Identity matrix
eye = tf.eye(3)           # 3×3 identity matrix

# Fill with a specific value
filled = tf.fill((2, 2), 7)  # 2×2 matrix, all elements are 7

3. Creating Random Tensors

Example

# Uniformly distributed random numbers
uniform = tf.random.uniform((2, 2), minval=0, maxval=1)

# Normally distributed random numbers
normal = tf.random.normal((3, 3), mean=0, stddev=1)

# Random permutation
shuffled = tf.random.shuffle(tf.constant([1, 2, 3, 4, 5]))

Basic Operations on Tensors

1. Mathematical Operations

Example

a = tf.constant([[1, 2], [3, 4]])
b = tf.constant([[5, 6], [7, 8]])

# Element-wise addition
add = tf.add(a, b)        # Or use operator overloading a + b

# Element-wise multiplication
mul = tf.multiply(a, b)   # Or a * b

# Matrix multiplication
matmul = tf.matmul(a, b)  # Or a @ b

# Other mathematical operations
sqrt = tf.sqrt(tf.cast(a, tf.float32))  # Square root (requires conversion to float type)

2. Shape Operations

Example

tensor = tf.constant([[1, 2, 3], [4, 5, 6]])

# Get shape
shape = tensor.shape  # Returns (2, 3)

# Change shape (reshape)
reshaped = tf.reshape(tensor, (3, 2))  # Change to 3 rows and 2 columns

# Transpose
transposed = tf.transpose(tensor)  # Change to 3 rows and 2 columns

# Expand dimensions (expand_dims)
expanded = tf.expand_dims(tensor, axis=0)  # Shape changes from (2,3) to (1,2,3)

3. Indexing and Slicing

Example

tensor = tf.constant([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

# Get a single element
elem = tensor[1, 2]  # Get the element at row 2, column 3 (value is 6)

# Slicing operation
row = tensor[1, :]    # Get all elements in the 2nd row [4,5,6]
col = tensor[:, 1]    # Get all elements in the 2nd column [2,5,8]
sub = tensor[0:2, 1:] # Get rows 1-2, columns 2-3 [[2,3],[5,6]]

Tensor Broadcasting Mechanism

Broadcasting is an important mechanism in TensorFlow for handling operations on tensors of different shapes. It automatically expands the smaller tensor to match the shape of the larger tensor.

Broadcasting Rules

  1. Compare dimensions starting from the last dimension and moving forward.
  2. Two dimensions are compatible if they are equal, or one of them is 1, or one of them does not exist.
  3. Perform expansion by copying along missing or size-1 dimensions.

Broadcasting Examples

Example

# Add a vector (3,) and a scalar ()
a = tf.constant([1, 2, 3])
b = tf.constant(2)
c = a + b  # The result is [3,4,5]; b is broadcast to [2,2,2]

# Add a matrix (3,1) and a vector (3,)
d = tf.constant([[1], [2], [3]])
e = tf.constant([10, 20, 30])
f = d + e  # d is broadcast to [[1,1,1],[2,2,2],[3,3,3]]
           # The result is [[11,21,31],[12,22,32],[13,23,33]]

Tensor Aggregation Operations

Common Aggregation Functions

Example

tensor = tf.constant([[1, 2, 3], [4, 5, 6]])

# Sum
sum_all = tf.reduce_sum(tensor)        # Sum of all elements → 21
sum_axis0 = tf.reduce_sum(tensor, 0)  # Sum along dimension 0 (rows) → [5,7,9]
sum_axis1 = tf.reduce_sum(tensor, 1)  # Sum along dimension 1 (columns) → [6,15]

# Compute mean
mean_all = tf.reduce_mean(tensor)      # Mean of all elements → 3.5

# Maximum/Minimum
max_val = tf.reduce_max(tensor)        # Maximum → 6
min_val = tf.reduce_min(tensor)       # Minimum → 1

# Logical operations
any_true = tf.reduce_any(tensor > 4)   # Whether any element > 4 → True
all_true = tf.reduce_all(tensor > 0)   # Whether all elements > 0 → True

Hands-on Practice

Exercise 1: Creating and Manipulating Tensors

Example

# 1. Create a 3×3 random matrix with element values between 0 and 10
random_matrix = tf.random.uniform((3, 3), minval=0, maxval=10, dtype=tf.int32)

# 2. Compute the transpose of the matrix
transposed_matrix = tf.transpose(random_matrix)

# 3. Compute the product of the matrix and its transpose
product = tf.matmul(random_matrix, transposed_matrix)

# 4. Compute the sum of the diagonal elements of the product matrix
diag_sum = tf.reduce_sum(tf.linalg.diag_part(product))

Exercise 2: Applying Broadcasting Mechanism

Example

# 1. Create a 4×1 matrix and a 1×4 vector
matrix = tf.constant([[1], [2], [3], [4]])
vector = tf.constant([10, 20, 30, 40])

# 2. Use the broadcasting mechanism to compute their sum
broadcast_sum = matrix + vector

# 3. Verify the shape and values of the result
print("Shape:", broadcast_sum.shape)  # Should be (4,4)
print("Result:", broadcast_sum.numpy())
Other Extensions