DocType Fixtures & Data Migration

← Frappe Cookbook

Problem: Ship default data with your app — tax templates, roles, workflows, ScoopJoy’s product categories — and handle schema changes safely as the app moves across versions and sites.

Solution: Declare fixtures in hooks.py for repeatable configuration data, and write patch files for one-time schema and data migrations. Fixtures re-import on every bench migrate; patches run exactly once per site, in order.

Step 1: Declare fixtures in hooks.py

List the DocTypes whose records you want to ship. A bare string exports all records of that DocType; a dict with dt and filters exports only the matching subset — use it to ship only ScoopJoy-specific records and avoid dragging in unrelated site data.

scoopjoy/hooks.py

# Export all records of these DocTypes with every `bench export-fixtures`
fixtures = [
    # Export ALL records
    "SJ Product Category",
    "Role",

    # Export with filters — only ScoopJoy-specific records
    {
        "dt": "Custom Field",
        "filters": [["module", "=", "ScoopJoy"]],
    },
    {
        "dt": "Property Setter",
        "filters": [["module", "=", "ScoopJoy"]],
    },
    {
        "dt": "Workflow",
        "filters": [["name", "in", ["SJ Outlet Approval", "SJ Agreement Approval"]]],
    },
    {
        "dt": "Workflow State",
        "filters": [["name", "like", "SJ%"]],
    },
    {
        "dt": "Workflow Action Master",
        "filters": [["name", "like", "SJ%"]],
    },
    {
        "dt": "Print Format",
        "filters": [["module", "=", "ScoopJoy"]],
    },
    {
        "dt": "Notification",
        "filters": [["module", "=", "ScoopJoy"]],
    },
]

Step 2: Export and import fixtures

Exporting writes one JSON file per DocType into your app’s fixtures/ folder, which you commit to version control. Importing happens automatically during bench migrate — after patches run.

scoopjoy/scoopjoy/fixtures/

# Export fixtures to JSON files in your app directory
bench --site scoopjoy.localhost export-fixtures --app scoopjoy

# Import fixtures (happens automatically during `bench migrate`)
# To force re-import:
bench --site scoopjoy.localhost import-doc scoopjoy/scoopjoy/fixtures/

# During `bench migrate`, fixtures are imported AFTER patches run.
# Order: patches.txt → schema sync → fixtures import

The exported JSON sits alongside your DocType definitions:

• Directoryapps/scoopjoy/scoopjoy/

  • hooks.py the fixtures list lives here
  • Directoryfixtures/

    • sj_product_category.json one file per exported DocType
    • role.json
    • custom_field.json
    • workflow.json
  • patches.txt the ordered patch registry
  • Directorypatches/

    • Directoryv1_1/

      • …
    • Directoryv1_2/

      • …

Step 3: Register migration patches in patches.txt

Each line is a dotted path to a patch function’s module. Patches run top to bottom, exactly once per site, recorded so they never re-run.

scoopjoy/patches.txt

# Each line is a dotted path to a patch function.
# Patches run in order, exactly once per site.
# Format: app_name.patches.folder.filename

# v1.1 patches
scoopjoy.patches.v1_1.rename_city_field
scoopjoy.patches.v1_1.migrate_outlet_status
scoopjoy.patches.v1_1.add_default_categories

# v1.2 patches
scoopjoy.patches.v1_2.change_revenue_field_type
scoopjoy.patches.v1_2.populate_city_code

Patch: rename a field

rename_field does the whole job — it renames the DB column, updates the DocType JSON, and fixes any Property Setters or Custom Fields that referenced the old name. Guard it with a has_column check so it’s idempotent.

scoopjoy/scoopjoy/patches/v1_1/rename_city_field.py

import frappe
from frappe.model.utils.rename_field import rename_field


def execute():
    """Rename 'city_name' to 'city' on SJ Franchise Outlet.

    This patch runs during `bench migrate` BEFORE schema sync (pre_model_sync)
    if placed before the model sync marker, or AFTER if placed after.

    rename_field handles:
    - Renaming the DB column (ALTER TABLE)
    - Updating the DocType JSON
    - Updating Property Setters
    - Updating Custom Fields referencing this field
    """
    doctype = "SJ Franchise Outlet"

    # Safety check: only run if old field still exists
    if frappe.db.has_column(doctype, "city_name"):
        rename_field(doctype, "city_name", "city")
        frappe.db.commit()

Patch: change a field type

Schema sync handles the column type change itself (Currency to Float). The patch is only for transforming the existing data when you need to.

scoopjoy/scoopjoy/patches/v1_2/change_revenue_field_type.py

import frappe


def execute():
    """Change monthly_revenue from Currency to Float.

    Steps:
    1. Update the field type in DocType JSON (handled by schema sync)
    2. This patch migrates existing data if needed
    """
    doctype = "SJ Franchise Outlet"

    # Check if migration is needed
    if not frappe.db.has_column(doctype, "monthly_revenue"):
        return

    # For Currency → Float, MariaDB handles the type cast automatically.
    # But if you need to transform the data (e.g., divide by 100 to remove paise):
    frappe.db.sql(
        """
        UPDATE `tabSJ Franchise Outlet`
        SET monthly_revenue = ROUND(monthly_revenue, 2)
        WHERE monthly_revenue IS NOT NULL
        """
    )
    frappe.db.commit()

Patch: migrate data between fields

A post_model_sync patch runs after the new column exists, so it can backfill the new city_code field from the existing city value. Note update_modified=False so the backfill doesn’t bump every outlet’s modified timestamp.

scoopjoy/scoopjoy/patches/v1_2/populate_city_code.py

import frappe


CITY_CODE_MAP = {
    "Mumbai": "MUM",
    "Delhi": "DEL",
    "Bangalore": "BLR",
    "Chennai": "CHE",
    "Hyderabad": "HYD",
    "Pune": "PUN",
    "Ahmedabad": "AMD",
    "Kolkata": "KOL",
    # … (rest of the map unchanged)
}


def execute():
    """Populate the new city_code field from the existing city field.

    This is a post_model_sync patch — the new city_code column already
    exists by the time this runs.
    """
    outlets = frappe.get_all(
        "SJ Franchise Outlet",
        filters={"city_code": ["is", "not set"]},
        fields=["name", "city"],
    )

    for outlet in outlets:
        code = CITY_CODE_MAP.get(outlet.city)
        if code:
            frappe.db.set_value(
                "SJ Franchise Outlet",
                outlet.name,
                "city_code",
                code,
                update_modified=False,
            )

    frappe.db.commit()
    frappe.msgprint(f"Populated city_code for {len(outlets)} outlets.")

Patch: seed default data

Seed data belongs in a patch, not in fixtures — the frappe.db.exists guard makes it idempotent so it never clobbers categories an outlet manager has since edited.

scoopjoy/scoopjoy/patches/v1_1/add_default_categories.py

import frappe


DEFAULT_CATEGORIES = [
    {"category_name": "All Products", "is_group": 1, "parent_sj_product_category": ""},
    {"category_name": "Ice Cream", "is_group": 1, "parent_sj_product_category": "All Products"},
    {"category_name": "Premium Ice Cream", "is_group": 0, "parent_sj_product_category": "Ice Cream"},
    {"category_name": "Classic Ice Cream", "is_group": 0, "parent_sj_product_category": "Ice Cream"},
    {"category_name": "Sugar-Free", "is_group": 0, "parent_sj_product_category": "Ice Cream"},
    {"category_name": "Frozen Desserts", "is_group": 1, "parent_sj_product_category": "All Products"},
    {"category_name": "Kulfi", "is_group": 0, "parent_sj_product_category": "Frozen Desserts"},
    {"category_name": "Sorbet", "is_group": 0, "parent_sj_product_category": "Frozen Desserts"},
    {"category_name": "Toppings & Add-ons", "is_group": 1, "parent_sj_product_category": "All Products"},
    {"category_name": "Sauces", "is_group": 0, "parent_sj_product_category": "Toppings & Add-ons"},
    {"category_name": "Nuts & Crunch", "is_group": 0, "parent_sj_product_category": "Toppings & Add-ons"},
]


def execute():
    """Seed default product categories if they don't exist."""
    for cat in DEFAULT_CATEGORIES:
        if not frappe.db.exists("SJ Product Category", cat["category_name"]):
            doc = frappe.get_doc({
                "doctype": "SJ Product Category",
                **cat,
            })
            doc.flags.ignore_permissions = True
            doc.insert()

    frappe.db.commit()

Why this works

Fixtures handle repeatable data that should be identical across all sites (templates, workflows, custom fields). Patches handle one-time data transformations that must run during upgrades. The patches.txt file ensures each patch runs exactly once, in order — so reruns of bench migrate are safe.

Common mistake

Putting data seed operations in fixtures instead of patches. Fixtures re-import on every bench migrate, which can overwrite user changes. Use fixtures for configuration (Custom Fields, Workflows) and patches for seed data that users might modify.