PyTorch torch.nn.Sigmoid Function
PyTorch torch.nn Reference Manual
torch.nn.SigmoidIt is the sigmoid activation function in PyTorch.
It maps input values to between 0 and 1, and is commonly used for binary classification or as a gate control signal.
Function Definition
torch.nn.Sigmoid()
Mathematical Principle
Sigmoid(x) = 1 / (1 + e^(-x))
Usage Examples
Example 1: Basic Usage
Example
import torch
import torch.nn as nn
sigmoid = nn.Sigmoid()
x = torch.tensor([-2.0, -1.0, 0.0, 1.0, 2.0])
output = sigmoid(x)
print("Input:", x.tolist())
print("Output:", output.tolist())
import torch.nn as nn
sigmoid = nn.Sigmoid()
x = torch.tensor([-2.0, -1.0, 0.0, 1.0, 2.0])
output = sigmoid(x)
print("Input:", x.tolist())
print("Output:", output.tolist())
Example 2: Binary Classification Output
Example
import torch
import torch.nn as nn
model = nn.Linear(10, 1)
sigmoid = nn.Sigmoid()
logits = model(torch.randn(4, 10))
probabilities = sigmoid(logits)
print("Logits:", logits.squeeze().tolist())
print("Probability:", probabilities.squeeze().tolist())
print("Prediction:", (probabilities > 0.5).squeeze().tolist())
import torch.nn as nn
model = nn.Linear(10, 1)
sigmoid = nn.Sigmoid()
logits = model(torch.randn(4, 10))
probabilities = sigmoid(logits)
print("Logits:", logits.squeeze().tolist())
print("Probability:", probabilities.squeeze().tolist())
print("Prediction:", (probabilities > 0.5).squeeze().tolist())
Example 3: nn.functional Version
Example
import torch
import torch.nn.functional as F
x = torch.randn(4, 10)
output = torch.sigmoid(x)
output2 = F.sigmoid(x)
print("Shape:", output.shape)
print("Both methods produce the same result:", torch.allclose(output, output2))
import torch.nn.functional as F
x = torch.randn(4, 10)
output = torch.sigmoid(x)
output2 = F.sigmoid(x)
print("Shape:", output.shape)
print("Both methods produce the same result:", torch.allclose(output, output2))
Usage Scenarios
- Binary Classification: output probability
- Gating Mechanism: control information flow
- Probability Output: scenarios requiring a 0-1 range
Note: When training deep networks, ReLU works better; Sigmoid tends to cause vanishing gradients.
Other Extensions