Python Pickle Module

In Python development, we often need to save runtime objects, or restore the previous state after the program restarts, for example:

Cache calculation results to disk to avoid repeated calculations
Save user configuration and program intermediate state
Pass complex objects between different Python processes

pickleThe module is what Python provides to solve these problemsOfficial built-in solution。

Python'spicklemodule is a standard library module for serializing and deserializing Python objects.

Before starting to use Pickle, you need to understand two core concepts:

Serialization (Pickling): convert Python objects into byte sequences
Deserialization (Unpickling): convert byte sequences back into Python objects

pickleThe module can save almost all Python objects (such as lists, dictionaries, class instances, etc.) to files, or transmit them over the network, and then reload them when needed.

Why use the Pickle module?

Data persistence: Save Python objects to files so that the data can still be accessed after the program is closed.
Data transmission: Transmit Python objects over the network, for example, passing data in distributed systems.
Fast storage and loading：pickleThe module can efficiently handle complex data structures and is suitable for scenarios that require fast storage and loading.

Typical use cases of Pickle

pickleIt is very suitable for the following scenarios:

Local data persistence
Save and restore program running state
Intermediate calculation result caching
Python inter-process communication (IPC)
Saving machine learning models and feature data

Unsuitable scenarios:

Cross-language data exchange
Front-end and back-end interface data transmission
Deserialization of untrusted data sources

What objects does Pickle support?

1. Supported object types

Type	Supported?
int / float / bool / str	Supported
list / tuple / dict / set	Supported
None	Supported
Custom class instances	Supported
Nested structures	Supported

Unsupported or not recommended objects

Open file objects
socket, database connections
Operating system resources
Objects that depend on the state of the runtime environment

Import module

Using the Pickle module is very simple, just import it:

import pickle

Basic usage of the Pickle module

1. Serialize objects

Usepickle.dump()method can serialize Python objects and save them to a file.

Example

import pickle

# Create a Python object
data = {
'name': 'Alice',
'age': 25,
'hobbies': ['reading', 'traveling']
}

# Serialize the object and save it to a file
with open('data.pkl', 'wb') as file:
pickle.dump(data, file)

'wb'Indicates opening the file in binary write mode.
pickle.dump()willdataThe object is serialized and written to the file.

2. Deserialize objects

Usepickle.load()method can load and deserialize Python objects from a file.

Example

import pickle

# Load and deserialize objects from a file
with open('data.pkl', 'rb') as file:
loaded_data = pickle.load(file)

print(loaded_data)

'rb'Indicates opening the file in binary read mode.
pickle.load()Read the byte stream from the file and deserialize it into a Python object.

3. Serialize to byte string

If you don't want to save to a file, you can use pickle.dumps() to serialize the object into a byte string:

Example

import pickle

data = [1, 2, 3, 4, 5]

# Serialize to byte string
byte_data = pickle.dumps(data)
print(byte_data)
# Output similar to: b'\x80\x04\x95\x0f\x00\x00\x00...'

4. Deserialize from byte string

Use pickle.loads() to deserialize objects from a byte string:

Example

import pickle

byte_data = b'\x80\x04\x95\x0f\x00\x00\x00\x00\x00\x00\x00]\x94(K\x01K\x02K\x03K\x04K\x05e.'

# Deserialize from byte string
original_data = pickle.loads(byte_data)
print(original_data)
# Output: [1, 2, 3, 4, 5]

5. Types of objects that can be serialized

Pickle can serialize most Python objects, including:

Basic data types: integers, floats, strings, booleans, None
Collection types: lists, tuples, dictionaries, sets
Instances of custom classes
Functions and classes (with certain limitations)

Example

import pickle

# Serialize different types of data
numbers = [1, 2, 3]
text = "Hello, Pickle"
dictionary = {'key': 'value'}
tuple_data = (1, 2, 3)
set_data = {1, 2, 3}

# Put all data into a list
all_data = [numbers, text, dictionary, tuple_data, set_data]

# Serialize
with open('mixed_data.pkl', 'wb') as f:
pickle.dump(all_data, f)

# Deserialize
with open('mixed_data.pkl', 'rb') as f:
loaded_data = pickle.load(f)
print(loaded_data)

6. Serialize custom classes

Pickle can handle instances of custom classes well:

Example

import pickle

class Student:
def __init__(self, name, age, grade):
self.name = name
self.age = age
self.grade = grade

def __repr__(self):
return f"Student(name={self.name}, age={self.age}, grade={self.grade})"

# Create instance
student = Student("Li Si", 20, "Junior")

# Serialize
with open('student.pkl', 'wb') as f:
pickle.dump(student, f)

# Deserialize
with open('student.pkl', 'rb') as f:
loaded_student = pickle.load(f)
print(loaded_student)
# Output: Student(name=Li Si, age=20, grade=Junior)

7. Serialize multiple objects

You can call pickle.dump() multiple times to save multiple objects:

Example

import pickle

data1 = {'item': 'apple', 'count': 5}
data2 = ['banana', 'orange', 'grape']
data3 = 42

# Save multiple objects
with open('multiple.pkl', 'wb') as f:
pickle.dump(data1, f)
pickle.dump(data2, f)
pickle.dump(data3, f)

# Read multiple objects (order must be consistent)
with open('multiple.pkl', 'rb') as f:
loaded_data1 = pickle.load(f)
loaded_data2 = pickle.load(f)
loaded_data3 = pickle.load(f)

print(loaded_data1)
print(loaded_data2)
print(loaded_data3)

8. Practical application examples

Example 1: Save and load machine learning model configuration

Example

import pickle

# Simulate machine learning model configuration
model_config = {
'model_type': 'RandomForest',
'n_estimators': 100,
'max_depth': 10,
'trained_date': '2024-01-13',
'accuracy': 0.95
}

# Save configuration
with open('model_config.pkl', 'wb') as f:
pickle.dump(model_config, f)

# Load configuration
with open('model_config.pkl', 'rb') as f:
config = pickle.load(f)
print(f"Model type: {config['model_type']}")
print(f"Accuracy: {config['accuracy']}")

Example 2: Cache calculation results

Example

import pickle
import os

def expensive_computation(n):
"""Simulate a time-consuming calculation"""
result = sum(i ** 2 for i in range(n))
return result

def compute_with_cache(n, cache_file='cache.pkl'):
# Check whether the cache exists
if os.path.exists(cache_file):
with open(cache_file, 'rb') as f:
cache = pickle.load(f)
if n in cache:
print("Read result from cache")
return cache[n]
else:
cache = {}

# Perform the calculation
print("Performing calculation...")
result = expensive_computation(n)

# Save to cache
cache[n] = result
with open(cache_file, 'wb') as f:
pickle.dump(cache, f)

return result

# Usage example
print(compute_with_cache(1000000)) # First time will calculate
print(compute_with_cache(1000000)) # Second time read from cache

Notes on the Pickle module

Security：pickleThe module executes arbitrary code during deserialization, so do not loadpickledata from untrusted sources, to avoid malicious attacks.
Compatibility：pickleThe generated byte stream is Python-specific, and there may be compatibility issues between different versions of Python.
Performance: For large data sets,pickleserialization and deserialization may be relatively slow; you may consider using more efficient serialization tools, such asjsonormsgpack。

Advanced usage: serialization of custom objects

pickleThe module supports serialization of custom classes. By default,pickleit saves the object's attributes and class name. If you need more complex serialization logic, you can implement__getstate__()and__setstate__()methods.

Example

import pickle

class Person:
def __init__(self, name, age):
self.name = name
self.age = age

def __getstate__(self):
# Custom serialization logic
return {'name': self.name, 'age': self.age}

def __setstate__(self, state):
# Custom deserialization logic
self.name = state['name']
self.age = state['age']

# Create object and serialize
person = Person('Bob', 30)
with open('person.pkl', 'wb') as file:
pickle.dump(person, file)

# Deserialize object
with open('person.pkl', 'rb') as file:
loaded_person = pickle.load(file)

print(loaded_person.name, loaded_person.age)

Common methods of the pickle module

Method	Description	Example
`pickle.dump(obj, file)`	Serialize object and write to file	`pickle.dump(data, open('data.pkl', 'wb'))`
`pickle.load(file)`	Read from file and deserialize object	`data = pickle.load(open('data.pkl', 'rb'))`
`pickle.dumps(obj)`	Serialize object to a byte string	`bytes_data = pickle.dumps([1, 2, 3])`
`pickle.loads(bytes)`	Deserialize object from a byte string	`lst = pickle.loads(bytes_data)`
`pickle.HIGHEST_PROTOCOL`	Highest available protocol version (attribute)	`pickle.dump(..., protocol=pickle.HIGHEST_PROTOCOL)`
`pickle.DEFAULT_PROTOCOL`	Default protocol version (attribute, usually 4)	`pickle.dumps(obj, protocol=pickle.DEFAULT_PROTOCOL)`

1. Serialize object to file

Example

import pickle
data = {'name': 'Alice', 'age': 25}
with open('data.pkl', 'wb') as f:
pickle.dump(data, f, protocol=pickle.HIGHEST_PROTOCOL)

2. Deserialize from file

Example

with open('data.pkl', 'rb') as f:
loaded_data = pickle.load(f)
print(loaded_data) # Output: {'name': 'Alice', 'age': 25}

3. Serialize to byte string (network transmission/cache)

Example

bytes_data = pickle.dumps([1, 2, 3], protocol=4)
restored_list = pickle.loads(bytes_data)

Protocol versions of the pickle module

Protocol version	Description
0	Human-readable ASCII format (compatible with older versions)
1	Binary format (compatible with older versions)
2	Python 2.3+ optimized support for class objects
3	Python 3.0+ default protocol (not supported in Python 2)
4	Python 3.4+ supports larger objects and more data types
5	Python 3.8+ supports memory optimization and data sharing

Other extensions

Python3Advanced Tutorial

Python Pickle Module

Why use the Pickle module?

Typical use cases of Pickle

What objects does Pickle support?

1. Supported object types

Unsupported or not recommended objects

Import module

Basic usage of the Pickle module

1. Serialize objects

Example

2. Deserialize objects

Example

3. Serialize to byte string

Example

4. Deserialize from byte string

Example

5. Types of objects that can be serialized

Example

6. Serialize custom classes

Example

7. Serialize multiple objects

Example

8. Practical application examples

Example

Example

Notes on the Pickle module

Advanced usage: serialization of custom objects

Example

Common methods of the pickle module

Example

Example

Example

Protocol versions of the pickle module