Python Serialization Tutorial with Examples

Serialization in Python is the process of converting an object into a byte stream or text format so that it can be easily saved to a file, sent over a network, or stored in a database.

The reverse process is called deserialization, which converts the byte stream back into a Python object.

Serialization is useful when you need to persist data, send data between systems, or store objects in files.

In this tutorial, we will cover:

1. What is Serialization?
2. Why is Serialization Important?
3. Serialization Using pickle
4. Serialization Using json
5. Serialization Using yaml
6. Handling Custom Python Objects in Serialization
7. Examples and Use Cases

Let’s explore these concepts with examples!

1. What is Serialization?

Serialization is the process of converting complex data structures (such as Python objects, lists, dictionaries) into a format that can be stored, transferred, or transmitted easily.

This process allows the serialized data to be reconstructed later via deserialization.

Python offers multiple libraries for serialization, including:

• pickle: Serializes objects into binary format.
• json: Serializes objects into a text-based format (JSON).
• yaml: A human-readable format often used for configuration files.

2. Why is Serialization Important?

Serialization is essential for several use cases:

• Saving objects to files for persistence across program runs.
• Transferring data over networks (e.g., for APIs, distributed systems).
• Storing objects in databases or sending data to external services.
• Sharing complex data between different programming languages or systems.

3. Serialization Using pickle

pickle is the standard Python library for serializing and deserializing Python objects to and from binary format. It can serialize almost any Python object, including complex types like classes and functions.

Example 1: Basic Serialization with pickle

import pickle

# Create a Python object (dictionary)
data = {"name": "Alice", "age": 30, "job": "Developer"}

# Serialize the object to a binary format
with open("data.pickle", "wb") as f:
    pickle.dump(data, f)

# Deserialize the object from the binary file
with open("data.pickle", "rb") as f:
    loaded_data = pickle.load(f)

print("Deserialized Data:", loaded_data)

Explanation:

• The dictionary data is serialized into a binary format using pickle.dump().
• The serialized data is saved to a file (data.pickle).
• The object is deserialized back into a Python dictionary using pickle.load().

Example 2: Serializing Custom Python Objects with pickle

import pickle

# Define a custom class
class Person:
    def __init__(self, name, age):
        self.name = name
        self.age = age

    def __repr__(self):
        return f"Person(name={self.name}, age={self.age})"

# Create an object of the Person class
person = Person("Bob", 25)

# Serialize the custom object
with open("person.pickle", "wb") as f:
    pickle.dump(person, f)

# Deserialize the custom object
with open("person.pickle", "rb") as f:
    loaded_person = pickle.load(f)

print("Deserialized Object:", loaded_person)

Explanation:

• The Person class is defined, and an object of this class is serialized using pickle.dump().
• The serialized object is saved to a file and later deserialized with pickle.load(), restoring the object to its original state.

4. Serialization Using json

The json module is used to serialize Python objects into JSON format (a text-based format that is widely used in web development and data interchange). However, JSON only supports basic data types like strings, numbers, lists, and dictionaries. It cannot serialize custom Python objects by default.

Example 3: Basic Serialization with json

import json

# Create a Python object (dictionary)
data = {"name": "Alice", "age": 30, "job": "Developer"}

# Serialize the object to a JSON format
with open("data.json", "w") as f:
    json.dump(data, f)

# Deserialize the object from the JSON file
with open("data.json", "r") as f:
    loaded_data = json.load(f)

print("Deserialized Data:", loaded_data)

Explanation:

• The dictionary data is serialized into JSON format using json.dump().
• The JSON data is saved to a file (data.json).
• The object is deserialized back into a Python dictionary using json.load().

Example 4: Handling Custom Objects in json

By default, json cannot serialize custom Python objects. You need to define a custom serialization and deserialization mechanism.

import json

# Define a custom class
class Person:
    def __init__(self, name, age):
        self.name = name
        self.age = age

# Custom encoder for the Person class
def person_encoder(obj):
    if isinstance(obj, Person):
        return {"name": obj.name, "age": obj.age}
    raise TypeError(f"Object of type {obj.__class__.__name__} is not serializable")

# Custom decoder for the Person class
def person_decoder(dct):
    if "name" in dct and "age" in dct:
        return Person(dct["name"], dct["age"])
    return dct

# Create an object of the Person class
person = Person("Charlie", 28)

# Serialize the custom object to JSON
with open("person.json", "w") as f:
    json.dump(person, f, default=person_encoder)

# Deserialize the custom object from JSON
with open("person.json", "r") as f:
    loaded_person = json.load(f, object_hook=person_decoder)

print("Deserialized Object:", loaded_person)

Explanation:

• The person_encoder function defines how to convert a Person object into a JSON serializable format (dictionary).
• The person_decoder function defines how to convert the dictionary back into a Person object during deserialization.

5. Serialization Using yaml

YAML is a human-readable data format that is often used for configuration files. The PyYAML library allows you to serialize Python objects into YAML format.

Example 5: Basic Serialization with yaml

import yaml

# Create a Python object (dictionary)
data = {"name": "Alice", "age": 30, "job": "Developer"}

# Serialize the object to YAML format
with open("data.yaml", "w") as f:
    yaml.dump(data, f)

# Deserialize the object from the YAML file
with open("data.yaml", "r") as f:
    loaded_data = yaml.safe_load(f)

print("Deserialized Data:", loaded_data)

Explanation:

• The dictionary data is serialized into YAML format using yaml.dump().
• The YAML data is saved to a file (data.yaml).
• The object is deserialized back into a Python dictionary using yaml.safe_load().

6. Handling Custom Python Objects in Serialization

Some serialization formats (like json and yaml) do not support custom Python objects by default. You can handle custom object serialization by:

1. Defining custom encoders and decoders (as shown with json).
2. Using pickle if you need to serialize complex Python objects, such as classes and functions, since it handles custom objects out of the box.

Example 6: Serializing and Deserializing Nested Objects with pickle

import pickle

# Define two custom classes
class Address:
    def __init__(self, city, country):
        self.city = city
        self.country = country

    def __repr__(self):
        return f"Address(city={self.city}, country={self.country})"

class Person:
    def __init__(self, name, age, address):
        self.name = name
        self.age = age
        self.address = address

    def __repr__(self):
        return f"Person(name={self.name}, age={self.age}, address={self.address})"

# Create objects with nested relationships
address = Address("New York", "USA")
person = Person("David", 35, address)

# Serialize the person object (which contains an address)
with open("nested_person.pickle", "wb") as f:
    pickle.dump(person, f)

# Deserialize the person object
with open("nested_person.pickle", "rb") as f:
    loaded_person = pickle.load(f)

print("Deserialized Nested Object:", loaded_person)

Explanation:

• The Person class has a nested Address object.
• pickle is used to serialize and deserialize the entire object structure, including nested objects, without needing custom encoders or decoders.

7. Examples and Use Cases

Example 7: Saving Application Settings to JSON

You can serialize application settings (such as configuration files or user preferences) to JSON for easy storage and retrieval.

import json

# Application settings
settings = {
    "theme": "dark",
    "font_size": 12,
    "show_line_numbers": True
}

#

 Save settings to a JSON file
with open("settings.json", "w") as f:
    json.dump(settings, f)

# Load settings from the JSON file
with open("settings.json", "r") as f:
    loaded_settings = json.load(f)

print("Loaded Settings:", loaded_settings)

Explanation:

• The application settings are saved to a JSON file.
• The settings can be loaded back into the application for later use.

Example 8: Sending Serialized Data Over a Network (Using JSON)

You can serialize data to JSON format and send it over a network (e.g., via HTTP requests in a REST API).

import json
import requests

# Data to be sent to a web API
data = {
    "username": "user1",
    "password": "securepassword"
}

# Serialize data to JSON
json_data = json.dumps(data)

# Send data as part of an HTTP POST request (example URL)
response = requests.post("https://example.com/api/login", data=json_data, headers={"Content-Type": "application/json"})

# Print response from the server
print(response.text)

Explanation:

• The data is serialized into JSON format using json.dumps().
• The serialized JSON is sent over the network using the requests library in a POST request.

Summary of Key Concepts for Python Serialization

Conclusion

In Python, serialization is a crucial technique for saving objects, transferring data between systems, and persisting program state.

In this tutorial, we explored:

• pickle for binary serialization of complex Python objects.
• json for serializing Python objects into the widely-used JSON format.
• yaml for human-readable serialization, useful for configuration files.
• Handling custom Python objects in serialization with encoders and decoders.