Part III · The ORM

Chapter 12: Migrations

Migrations are how a change to a model becomes a change to a database. A migration file lists operations; makemigrations writes such files by comparing the models with what the existing files already describe, and migrate applies them to a database in the order their dependencies allow, recording each one there in the table "django_migrations".

A model class says what a table should look like. It does not say what the table looks like at this moment in any particular database, and the two drift apart each time a model is edited: on a developer’s machine, on a colleague’s, in production, each at its own time. Django’s answer is to keep the history of the schema in the project, as code. Each change is written down once, as a migration file in the application’s migrations package, and committed beside the models it describes. A database is brought up to date by running the files it has not yet had.

So the facts are kept in separate places. The models say what the schema should be now. The migration files say how it came to be so, one step after another, and therefore also what it was after each step. And each database says, in one table of its own, which of the steps it has had. makemigrations reads the models and the files and writes new files; it sends a database nothing that changes it. migrate reads the files and the table, and changes the database.

The example in this chapter is harbour, the shipping line of the chapters on the ORM, with its three applications: fleet (ports, ships, voyages), crew (sailors and officers) and office (licences, manifests, berths, logbooks); Models and fields shows their models. A fourth is added to it here, yard, a repair yard, because a history needs something that changes: yard’s models are edited several times in the course of the chapter. They begin like this.

# harbour/yard/models.py, less its docstrings
from django.db import models


class Dock(models.Model):
    name = models.CharField(max_length=40)
    port = models.ForeignKey("fleet.Port", on_delete=models.CASCADE)


class Repair(models.Model):
    ship = models.ForeignKey("fleet.Ship", on_delete=models.CASCADE)
    dock = models.ForeignKey(Dock, on_delete=models.PROTECT)
    started = models.DateField()
    notes = models.TextField(blank=True)

In this chapter

12.1 A migration: its operations, its dependencies and how it is applied A migration, as the loader makes it, is an instance of the class Migration that a migration file defines, a subclass of the Migration of django.db.migrations (Migration). It holds a list of operations and the keys of the migrations it depends on, and runs its operations in three ways: Migration.mutate_state runs their state halves alone, Migration.apply both halves forwards, and Migration.unapply the database halves backwards.
12.2 Operations: the base class and its two halves An operation is one change that a migration makes, an instance of a subclass of django.db.migrations.operations.base.Operation. It can make the change in two places: its state half, Operation.state_forwards, in a project state, and its database half, Operation.database_forwards and Operation.database_backwards, in a database. The base class also keeps the arguments an operation was made with, for Operation.deconstruct, and gives every operation its flags, its description and Operation.allow_migrate_model.
12.3 Project state: ProjectState and its model states A project state, a ProjectState, is a project’s models held as data: a dictionary of model states, each a ModelState with one model’s fields, options, bases and managers and no class (ProjectState, ModelState). The state half of an operation is given one to change, the autodetector compares two, and historical models are rendered from one.
12.4 Historical models: StateApps and rendering A historical model is a model class made from a model state: the model as it stood at some point of the project’s migrations. ProjectState.apps makes them all at its first read and keeps them in a StateApps, a registry that belongs to the project state. After that, the state half of an operation makes again only the model it changed and the models related to it, through ProjectState.reload_model or ProjectState.reload_models; a model removed is only taken out.
12.5 The model operations: CreateModel, DeleteModel, RenameModel and the option operations The model operations are the migration operations that act on a model as a whole: CreateModel and DeleteModel make and drop its table, RenameModel renames it and what is named after it, and the option operations (AlterModelTable, AlterModelTableComment, AlterUniqueTogether, AlterIndexTogether, AlterOrderWithRespectTo, AlterModelOptions, AlterModelManagers) each change options of its Meta or, the last, its managers. All are subclasses of ModelOperation (django/db/migrations/operations/models.py).
12.6 The field operations, and those on indexes and constraints AddField, RemoveField, AlterField and RenameField are the migration operations on one field of a model, and AddIndex, RemoveIndex, RenameIndex, AddConstraint, RemoveConstraint and AlterConstraint those of django.db.migrations on one index or constraint of its Meta. They differ in which project state each takes its model from, in what running backwards means, and in what preserve_default=False does with a default.
12.7 RunPython, RunSQL and SeparateDatabaseAndState RunPython calls a function with the historical models and the schema editor, RunSQL sends SQL of the author’s own, and SeparateDatabaseAndState gives the project state one list of operations and the database another. They are the classes of django/db/migrations/operations/special.py.
12.8 What has been applied: MigrationRecorder and the table "django_migrations" MigrationRecorder keeps, in the table "django_migrations" of one database, a row for each migration applied there. It makes the table itself, through a model the main registry never holds; MigrationRecorder.applied_migrations, MigrationRecorder.record_applied and MigrationRecorder.record_unapplied read and change what is applied.
12.9 The migration graph: MigrationGraph The migration graph, a MigrationGraph, has a node for each migration the loader keeps and an edge for each dependency the loader gives it, and it is what orders migrations. MigrationGraph.forwards_plan and MigrationGraph.backwards_plan walk it for an order in which migrations can be applied and unapplied, MigrationGraph.root_nodes and MigrationGraph.leaf_nodes find where an application’s history begins and ends, and MigrationGraph.make_state builds the project state at a node.
12.10 Finding migrations: MigrationLoader MigrationLoader is where migrations stop being files. MigrationLoader.load_disk imports each application’s migration modules and makes a migration of each; MigrationLoader.build_graph adds which of them one database has applied and joins them into the graph by their dependencies, "__first__" and "__latest__" among them.
12.11 Applying a plan: MigrationExecutor MigrationExecutor brings one database to a set of targets: MigrationExecutor.migration_plan works out which migrations must run and in which direction, and MigrationExecutor.migrate runs them, giving each migration its project state, opening a schema editor for it and recording it in "django_migrations". With fake=True the rows change and the operations do nothing to the database; with fake_initial=True alone, MigrationExecutor.detect_soft_applied looks in the database for what a migration would create.
12.12 The migrate command, showmigrations and sqlmigrate The migrate command is a front for the executor: it turns its arguments into targets, refuses a history it cannot work from, sends the signals pre_migrate and post_migrate around MigrationExecutor.migrate, and prints progress. showmigrations and sqlmigrate each read the same files and rows and record nothing: one lists the migrations and marks those applied, the other prints the statements one migration would send.
12.13 The autodetector: comparing two project states The autodetector, a MigrationAutodetector, is given two project states and finds operations that turn the first into the second. MigrationAutodetector._detect_changes sorts the models of both into kept, added and removed, then calls, in a fixed order, the methods that add operations to MigrationAutodetector.generated_operations; fields are compared through MigrationAutodetector.deep_deconstruct.
12.14 The autodetector: fields, indexes and constraints For a model that both project states have, MigrationAutodetector compares what the model is made of: its fields, its indexes, its constraints, and the options unique_together and order_with_respect_to. The differences become RenameField, AddField, RemoveField and AlterField, the operations on indexes and constraints, AlterUniqueTogether and AlterOrderWithRespectTo.
12.15 From operations to migrations: dependencies, order and names When its generators have run, the autodetector holds a flat list of operations for each application, each operation with the OperationDependency tuples that say what it waits for. MigrationAutodetector._sort_migrations orders each list, MigrationAutodetector._build_migration_list cuts the lists into migrations that depend on one another, and MigrationAutodetector.arrange_for_graph numbers and names them.
12.16 The questioner: MigrationQuestioner The questioner is the object the autodetector asks what the two project states it compares do not tell it; makemigrations asks one itself before a merge. MigrationQuestioner asks no one, InteractiveMigrationQuestioner asks the person at the terminal, and NonInteractiveMigrationQuestioner answers what it can and ends the command with status 3 where it cannot.
12.17 The optimizer: MigrationOptimizer and reduce The optimizer, a MigrationOptimizer, shortens a list of operations: a model created and then given a field becomes a model created with that field, and a field added and then removed becomes nothing. It knows no operation itself: it asks each one, through its method reduce, what to make of a later one, and operations answer with the help of references_model and references_field. The command optimizemigration runs it over one migration file.
12.18 Serializing values: serializer_factory and deconstruct A migration file is Python source that Django writes, so every value in an operation has to become an expression that makes the value again when the file is imported. serializer_factory picks a serializer for a value, the serializer returns the expression as a string with the import lines it needs, and an object with a deconstruct method is written as a call with the arguments the method returns.
12.19 Writing a migration file: MigrationWriter A MigrationWriter is given a migration and produces the text of a module that will define the same migration when it is imported, MigrationWriter.as_string, and the place on disk where that module belongs, MigrationWriter.path. Each operation in the text is written by an OperationWriter, and each value inside an operation by the serializer.
12.20 The makemigrations command The makemigrations command writes new migration files. Command.handle makes a MigrationLoader with no connection, has a MigrationAutodetector compare the project state the loaded migrations describe with the one read off the models, and gives each migration that comes back to a MigrationWriter. --merge, --empty and --update each leave that road at one point; --dry-run, --check and --scriptable change what is written, asked and printed along it (Command.handle).
12.21 Squashed migrations: replaces and the squashmigrations command A squashed migration is one migration that stands in for a run of others, which it names in Migration.replaces (Migration). squashmigrations writes one; MigrationLoader.replace_migration decides for each database whether the graph holds it or the migrations it replaces; and MigrationExecutor.record_migration and MigrationExecutor.check_replacements keep the rows of "django_migrations" in step with both.

A migration is a list of operations

The first makemigrations on this project that was given the labels of its four applications wrote a file for each, and a second for crew. This is yard’s, recorded as Django wrote it; each line of the file stands after a bar.

| # Generated by Django 6.2.dev20260926051415 on (the date and the time)
|
| import django.db.models.deletion
| from django.db import migrations, models
|
|
| class Migration(migrations.Migration):
|
|     initial = True
|
|     dependencies = [
|         ('fleet', '0001_initial'),
|     ]
|
|     operations = [
|         migrations.CreateModel(
|             name='Dock',
|             fields=[
|                 ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
|                 ('name', models.CharField(max_length=40)),
|                 ('port', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='fleet.port')),
|             ],
|         ),
|         migrations.CreateModel(
|             name='Repair',
|             fields=[
|                 ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
|                 ('started', models.DateField()),
|                 ('notes', models.TextField(blank=True)),
|                 ('dock', models.ForeignKey(on_delete=django.db.models.deletion.PROTECT, to='yard.dock')),
|                 ('ship', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='fleet.ship')),
|             ],
|         ),
|     ]

A migration file is a module that defines one class, named Migration, a subclass of the class of that name in django.db.migrations. In a file that Django writes the class has no methods, only attributes, and two of them carry nearly everything: dependencies, the migrations that must be applied before this one, and operations, what this one does (initial marks a migration that begins an application’s history, which matters to migrate --fake-initial). When Django loads the file it makes one instance of the class, giving it the module’s name and the application’s label. That instance is a migration. This one is written yard.0001_initial, and the pair ("yard", "0001_initial") is its key: what a dependency names, what the graph files it under, and what its row in the database holds. The dependency here is the key of fleet’s first migration: yard’s cannot run until fleet’s has, because both of its tables point at tables of fleet.

Each entry of operations is an operation, an instance of a subclass of Operation: here two of CreateModel, each holding a model’s name and its fields. An operation is one change, and it can make that change in two places. Its state half, Operation.state_forwards, changes Django’s own description of the models, in memory: after a CreateModel the description has one model more. Its database half, Operation.database_forwards, changes a database, which a CreateModel does by calling the schema editor of Database backends; Operation.database_backwards is what runs when the migration is unapplied. The halves are separate because they are needed separately. To know what the models were at some point of the history takes state halves only and touches no database.

An application for which Django finds no package of migrations is unmigrated. Harbour’s four began so: makemigrations with no application named made nothing, and made their first migrations when it was given their labels. migrate makes such an application’s tables only when asked to with --run-syncdb, straight from the models and with nothing recorded.

A migration: its operations, its dependencies and how it is applied is about the class and what it does with its operations, and Operations: the base class and its two halves about what every operation shares. The operations of django.db.migrations.operations are in three sections: The model operations: CreateModel, DeleteModel, RenameModel and the option operations, The field operations, and those on indexes and constraints, and RunPython, RunSQL and SeparateDatabaseAndState, three that makemigrations never chooses for itself.

The project state

Django’s description of the models is the project state, a ProjectState. It holds model states, each one model’s name, its fields as field objects that belong to no class, its options, its bases and its managers. It is data, not classes. Begin with an empty project state, run the state half of every operation of every migration in an order their dependencies allow, and the result is the models as the migration files describe them.

A project state can also be read straight off the model classes, by ProjectState.from_apps. makemigrations compares the two: the state the migration files make is its from-state, and the state read off the models its to-state.

Some users of a project state need classes: the schema editor is given a model class, and a function in a data migration wants to run queries. For them a project state is rendered: each model state is made into a model class, a historical model, and registered in a registry that belongs to the state, the state’s registry, apart from the main registry in which the models of the project live. A historical model has the fields and the options the model had at that point of the history. It does not have the methods written in the body of the model’s class.

Project state: ProjectState and its model states is about the data and how operations change it, and Historical models: StateApps and rendering about the classes made from it. With these names, the two commands are these.

Figure 12.1 The two commands. makemigrations compares two project states, one read off the models and one built from the migration files, and writes the difference as new files. migrate reads the files and the table of applied migrations, makes a plan, and for each migration of the plan runs its operations and writes its row.

Making migrations

makemigrations builds the two states, has them compared, and writes what the comparison finds. Here is the first one, recorded. In a recording the lines are what a running Django did. Where an order is the point, as here, the lines are calls, nested as they were made; only the calls the passage is about are written, and everything between them ran and is left out. A method is written by the class that declares it; where it was called on an object of a subclass, the line gives the object’s class first and the declaring class after it in parentheses. An arrow is followed by what a call returned. Square brackets after a call name the migration or the operation it was called on or, after the writer’s as_string, the migration being written. A line beginning SQL is a statement as Django handed it to its cursor; stdout | begins a line a command printed; and a line beginning python manage.py is a command line, run as a process of its own. Every recording of this chapter ran on SQLite.

makemigrations.Command.handle('fleet', 'crew', 'office', 'yard')
  MigrationLoader.__init__(connection=None, ignore_no_migrations=True)
    MigrationLoader.build_graph
  MigrationLoader.check_consistent_history(the connection "default")
  InteractiveMigrationQuestioner.__init__(specified_apps={'crew', 'fleet', 'office', 'yard'}, dry_run=False)
  MigrationLoader.project_state  ->  a ProjectState of 0 model states, its registry not rendered
  ProjectState.from_apps(the main registry)  ->  a ProjectState of 13 model states, its registry not rendered
  MigrationAutodetector.__init__(from_state=a ProjectState of 0 model states, its registry not rendered, to_state=a ProjectState of 13 model states, its registry not rendered, questioner=an InteractiveMigrationQuestioner)
  MigrationAutodetector.changes(graph=the graph (0 nodes), trim_to_apps={'crew', 'fleet', 'office', 'yard'}, convert_apps={'crew', 'fleet', 'office', 'yard'})
    MigrationAutodetector._detect_changes(convert_apps={'crew', 'fleet', 'office', 'yard'}, graph=the graph (0 nodes))  ->  {'crew': [crew.auto_1, crew.auto_2], 'fleet': [fleet.auto_1], 'office': [office.auto_1], 'yard': [yard.auto_1]}
    MigrationAutodetector.arrange_for_graph(graph=the graph (0 nodes))
      -> {'crew': [crew.0001_initial, crew.0002_initial], 'fleet': [fleet.0001_initial], 'office': [office.0001_initial], 'yard': [yard.0001_initial]}
  makemigrations.Command.write_migration_files(changes={'crew': [crew.0001_initial, crew.0002_initial], 'fleet': [fleet.0001_initial], 'office': [office.0001_initial], 'yard': [yard.0001_initial]})
    MigrationWriter.as_string  [crew.0001_initial]

The loader finds the migration files and builds from them the graph that orders them, a node for each migration and edges for their dependencies. There are no files yet, so the graph has no nodes and the from-state is empty. This loader is made with no connection: it does not ask which migrations a database has had. The check that follows it is a separate matter, that the history recorded in the database is not out of order. The to-state, read off the models of the four applications, has thirteen model states.

The autodetector is given the two and finds operations that turn the first into the second. It matches models by name and compares fields by the arguments they would be made again with. Where two states cannot settle a question it asks the questioner: the one this makemigrations was given puts the question to whoever ran the command, and the one it is given under --noinput answers from a default or stops the command. What the autodetector produces first is a list of operations for each application. It then has to order each list so that it can run, cut the lists into migrations, and give the migrations dependencies on each other.

Harbour shows why the cutting is needed. Among the fields of its models that point from one application into another, these make a circle:

# harbour/fleet/models.py, in Ship
captain = models.OneToOneField("crew.Sailor", null=True, blank=True, on_delete=models.SET_NULL, related_name="command")

# harbour/crew/models.py, in Sailor
ship = models.ForeignKey("fleet.Ship", on_delete=models.CASCADE, related_name="crew")
pilots_into = models.ManyToManyField("fleet.Port", through="office.Licence", related_name="pilots")

# harbour/office/models.py, in Licence
sailor = models.ForeignKey("crew.Sailor", on_delete=models.CASCADE)
port = models.ForeignKey("fleet.Port", on_delete=models.CASCADE)

A ship names a sailor and a sailor names a ship, so neither application can be created whole before the other. What _detect_changes returned in the recording, under names not yet final, is the way out: crew has two migrations. The first creates Sailor, and Officer with it, without the two fields that point out of crew; fleet’s migration and office’s can then run; crew’s second adds the two fields, and the proxy Veteran. MigrationAutodetector.arrange_for_graph then gives each migration its number and its name. The whole of this is in The autodetector: comparing two project states, The autodetector: fields, indexes and constraints, From operations to migrations: dependencies, order and names and The questioner: MigrationQuestioner. Before a migration leaves the autodetector its operations go through the optimizer, which merges operations that can be said as one (The optimizer: MigrationOptimizer and reduce).

The writer turns each migration into the text of a module, once for each file; the quote stops at the first. That means turning every value inside every operation, each field with its arguments, into Python source that makes the value again when the file is imported; a value that Django cannot write in that way cannot go into a migration file (Serializing values: serializer_factory and deconstruct, Writing a migration file: MigrationWriter). The command around all of it is in The makemigrations command.

The graph

The five migrations and their dependencies make the graph, in which an edge runs from a node to each of its parents, the migrations it depends on. The numbers in the files’ names make no migration depend on another; the graph’s edges do. MigrationGraph.forwards_plan gives, for a node, everything that must be applied before it, in an order that can be followed, ending with the node:

graph.forwards_plan(('crew', '0002_initial'))  ->  [('crew', '0001_initial'), ('fleet', '0001_initial'), ('office', '0001_initial'), ('crew', '0002_initial')]

A node on which no migration of its own application depends is a leaf node: the newest migration of its application. The migration graph: MigrationGraph is about the structure and its walks, and Finding migrations: MigrationLoader about how it is filled from the files.

Applying migrations

The first migrate, on an empty database, begins by working out what to do.

migrate.Command.handle()
  MigrationExecutor.__init__(connection=the connection "default", progress_callback=the method Command.migration_progress_callback of a Command)
    MigrationLoader.__init__(connection=the connection "default")
      MigrationLoader.build_graph
        MigrationRecorder.has_table
          SQL SELECT name, type FROM sqlite_master WHERE type in ('table', 'view') AND NOT name='sqlite_sequence' ORDER BY name
          -> False
  MigrationLoader.check_consistent_history(the connection "default")
    MigrationRecorder.has_table
      SQL SELECT name, type FROM sqlite_master WHERE type in ('table', 'view') AND NOT name='sqlite_sequence' ORDER BY name
      -> False
  MigrationLoader.detect_conflicts  ->  {}
  MigrationExecutor.migration_plan(targets=[('crew', '0002_initial'), ('fleet', '0001_initial'), ('office', '0001_initial'), ('yard', '0001_initial')])  ->  [(crew.0001_initial, False), (fleet.0001_initial, False), (office.0001_initial, False), (crew.0002_initial, False), (yard.0001_initial, False)]
...
  emit_pre_migrate_signal
...
  MigrationExecutor.migrate(targets=[('crew', '0002_initial'), ('fleet', '0001_initial'), ('office', '0001_initial'), ('yard', '0001_initial')], plan=a plan of 5, state=a ProjectState of 0 model states, its registry rendered, fake=False, fake_initial=False)
    MigrationRecorder.ensure_schema

The executor, a MigrationExecutor, is made for one connection, and it makes a loader with that connection. This loader reads the files as before and also asks the database which migrations have been applied there: the recorder looks for the table "django_migrations", finds none, and so nothing is applied. The command then refuses two conditions it cannot work from, a recorded history that is out of order and an application with two leaf nodes, and neither holds. With no arguments the targets are all the leaf nodes, and MigrationExecutor.migration_plan turns targets into a plan: a list of pairs, a migration and whether it is to run backwards. Here all five run forwards, in an order taken from the graph. The command has the executor make the project state of what is already applied, which here is nothing, renders it, and sends the signal pre_migrate for each application that has a models module. A clone of that state goes to MigrationExecutor.migrate, which has the recorder make its table and then carries out the plan.

MigrationExecutor._migrate_all_forwards(state=a ProjectState of 0 model states, its registry rendered, plan=a plan of 5, full_plan=a plan of 5, fake=False, fake_initial=False)
  MigrationExecutor.apply_migration(state=a ProjectState of 0 model states, its registry rendered, migration=crew.0001_initial)  (what it called is not written, nor the 9 statements it sent)
    -> a ProjectState of 2 model states, its registry rendered
  MigrationExecutor.apply_migration(state=a ProjectState of 2 model states, its registry rendered, migration=fleet.0001_initial)  (what it called is not written, nor the 27 statements it sent)
    -> a ProjectState of 5 model states, its registry rendered
  MigrationExecutor.apply_migration(state=a ProjectState of 5 model states, its registry rendered, migration=office.0001_initial)  (what it called is not written, nor the 16 statements it sent)
    -> a ProjectState of 10 model states, its registry rendered
  MigrationExecutor.apply_migration(state=a ProjectState of 10 model states, its registry rendered, migration=crew.0002_initial)  (what it called is not written, nor the 15 statements it sent)
    -> a ProjectState of 11 model states, its registry rendered
  MigrationExecutor.apply_migration(state=a ProjectState of 11 model states, its registry rendered, migration=yard.0001_initial)
    migrate.Command.migration_progress_callback(action='apply_start', migration=yard.0001_initial)
    DatabaseWrapper.schema_editor(atomic=True)  (BaseDatabaseWrapper.schema_editor)
    DatabaseSchemaEditor.__enter__
      DatabaseSchemaEditor.__enter__  (BaseDatabaseSchemaEditor.__enter__)
        SQL BEGIN
    Migration.apply(project_state=a ProjectState of 11 model states, its registry rendered, schema_editor=the schema editor)  [yard.0001_initial]
      CreateModel.state_forwards  [Create model Dock]
        ProjectState.add_model(the model state yard.Dock)
      CreateModel.database_forwards  [Create model Dock]
        DatabaseSchemaEditor.create_model(the historical model yard.Dock)  (BaseDatabaseSchemaEditor.create_model)
          SQL CREATE TABLE "yard_dock" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "name" varchar(40) NOT NULL, "port_id" bigint NOT NULL REFERENCES "fleet_port" ("id") DEFERRABLE INITIALLY DEFERRED)
      CreateModel.state_forwards  [Create model Repair]
        ProjectState.add_model(the model state yard.Repair)
      CreateModel.database_forwards  [Create model Repair]
        DatabaseSchemaEditor.create_model(the historical model yard.Repair)  (BaseDatabaseSchemaEditor.create_model)
          SQL CREATE TABLE "yard_repair" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "started" date NOT NULL, "notes" text NOT NULL, "dock_id" bigint NOT NULL REFERENCES "yard_dock" ("id") DEFERRABLE INITIALLY DEFERRED, "ship_id" bigint NOT NULL REFERENCES "fleet_ship" ("id") DEFERRABLE INITIALLY DEFERRED)
      -> a ProjectState of 13 model states, its registry rendered
    DatabaseSchemaEditor.__exit__
      DatabaseSchemaEditor.__exit__  (BaseDatabaseSchemaEditor.__exit__)
        SQL CREATE INDEX "yard_dock_port_id_205f1b55" ON "yard_dock" ("port_id")
        SQL CREATE INDEX "yard_repair_dock_id_e7d19dbe" ON "yard_repair" ("dock_id")
        SQL CREATE INDEX "yard_repair_ship_id_3fb74465" ON "yard_repair" ("ship_id")
        DatabaseWrapper.commit  (BaseDatabaseWrapper.commit)
    MigrationExecutor.record_migration(app_label='yard', name='0001_initial')
      MigrationRecorder.record_applied(app='yard', name='0001_initial')
        MigrationRecorder.ensure_schema
          MigrationRecorder.has_table  ->  True
        SQL INSERT INTO "django_migrations" ("app", "name", "applied") VALUES (%s, %s, %s) RETURNING "django_migrations"."id" with params ('yard', '0001_initial', '(the date and the time)')
    migrate.Command.migration_progress_callback(action='apply_success', migration=yard.0001_initial)

One project state is carried along the plan: each MigrationExecutor.apply_migration is given the state as the migrations before it left it and returns it with its own migration’s models in. For each migration the executor opens a schema editor, and Migration.apply goes through the operations in order. An operation’s state half runs first, here adding a model state to the project state. Its database half is then given the state before and the state after, takes the historical model from the state after, and hands it to BaseDatabaseSchemaEditor.create_model, which sends "CREATE TABLE". Here, on SQLite, which can roll back changes to its schema, the editor’s block was a transaction, begun as the editor was entered and committed as it was left (a migration that sets atomic = False gets none); the three indexes on yard’s foreign keys are statements the editor held back until then (The schema editor: BaseDatabaseSchemaEditor). The executor has the recorder write the migration’s row: here after the block, because the editor still held statements as the operations ended, and inside it where the editor holds none, so that the change and its row are committed together.

Applying a plan: MigrationExecutor has the plan, the two directions and what --fake does; The migrate command, showmigrations and sqlmigrate has the command’s options, the two signals, and the two commands that record nothing.

What a database remembers

After the first migrate the table "django_migrations" has five rows.

django_migrations: id 1, app 'crew', name '0001_initial', applied (the date and the time)
django_migrations: id 2, app 'fleet', name '0001_initial', applied (the date and the time)
django_migrations: id 3, app 'office', name '0001_initial', applied (the date and the time)
django_migrations: id 4, app 'crew', name '0002_initial', applied (the date and the time)
django_migrations: id 5, app 'yard', name '0001_initial', applied (the date and the time)

A migration is applied on a database when the table there has its row, and beside its id a row holds a label, a name and a time, no more. The rest follows from that. Each database has its own table, so each is migrated on its own. To unapply a migration is to run the database halves of its operations backwards, last operation first, and delete the row; migrate yard zero plans that for every applied migration of yard and for whatever depends on one. And migrate --fake writes or deletes rows and leaves the schema alone, which is how a database whose schema was changed by other means is brought back into step. What has been applied: MigrationRecorder and the table "django_migrations" is about the table and its one class.

A change to the models

Yard’s models are now edited.

The models of yard, second version: harbour/yard/models.py, each line put in marked + and each line taken out marked -; four changes at once: Dock.name made longer, Dock.metres added, Repair.started renamed begun, and an index on Repair
    |   from django.db import models
    |
    |
    |   class Dock(models.Model):
    |       """A dry dock of the repair yard, in one of the fleet's ports."""
    |
    | -     name = models.CharField(max_length=40)
    | +     name = models.CharField(max_length=80)
    |       port = models.ForeignKey("fleet.Port", on_delete=models.CASCADE)
    | +     metres = models.PositiveSmallIntegerField(default=120)
    |
    |
    |   class Repair(models.Model):
    |       """A ship in a dock, under repair."""
    |
    |       ship = models.ForeignKey("fleet.Ship", on_delete=models.CASCADE)
    |       dock = models.ForeignKey(Dock, on_delete=models.PROTECT)
    | -     started = models.DateField()
    | +     begun = models.DateField()
    |       notes = models.TextField(blank=True)
    | +
    | +     class Meta:
    | +         indexes = [models.Index(fields=["dock", "begun"], name="repair_by_dock")]

python manage.py makemigrations yard
  stdout | Was repair.started renamed to repair.begun (a DateField)? [y/N]
  stdin  | y
  stdout | Migrations for 'yard':
  stdout |   harbour/yard/migrations/0002_rename_started_repair_begun_dock_metres_and_more.py
  stdout |     ~ Rename field started on repair to begun
  stdout |     + Add field metres to dock
  stdout |     ~ Alter field name on dock
  stdout |     + Create index repair_by_dock on field(s) dock, begun of model repair

This time the from-state is not empty. The loader’s graph has five nodes, and running their state halves gives the thirteen models as they were; the to-state has yard’s as they are now. The four changes come out as four operations. One of them needed the question: the two states show that Repair has lost a date field called started and gained a date field called begun, and whether that is one column renamed or one dropped and another made is something only a person knows. The new migration depends on yard’s leaf node, takes the next number, and is named from its operations. A migrate would now find one migration with no row, and apply it.

Yard’s migrations

Yard’s history goes on from there.

Table 12.1 Yard’s migrations, in the order they were made.

number	how it came to be
0001	yard.0001_initial: the first makemigrations, Dock and Repair
0002	makemigrations after the edit shown here, which named it from its four operations
0003	yard.0003_repair_foreman: makemigrations after Repair gained a column that is not null and has no default; the default for the rows there were was typed at the prompt
0004	yard.0004_name_the_foremen: made empty by `makemigrations --empty` and filled in by hand with a `RunPython` and a `RunSQL`
0005	yard.0005_repair_by_foreman: written by hand, a second index in a `SeparateDatabaseAndState`
0006	yard.0006_dock_covered and yard.0006_long_dock, which both depend on yard.0005: one from makemigrations, one a colleague’s
0007	yard.0007_merge: `makemigrations --merge`, no operations and a dependency on both
squashed	yard.0001_squashed_0007_merge: squashmigrations, one migration that replaces the eight before it
0008	yard.0008_alter_repair_foreman: makemigrations after the squash
0009	yard.0009_repair_inspector: makemigrations after Repair gained a foreign key to the user model

Long histories

Two things happen to a history as it grows, and each has its machinery. When two branches of a project each add a migration to one application, the application has two leaf nodes; migrate refuses to go on, and so does makemigrations for that application, until makemigrations --merge has written a migration with no operations that depends on both (The makemigrations command). And when the files have become many, squashmigrations writes one replacing migration that does what a run of them did, with their operations put through the optimizer unless told otherwise; the loader then uses the replacing migration or the ones it replaces, according to which of their rows each database has, and counts the replacing one applied where it has them all (Squashed migrations: replaces and the squashmigrations command).

The neighbours

Migrations decide what is to change and in what order, and leave the changing to the schema editor, which writes the statements, and to the connection’s transactions (Database backends). The operations on models, fields, indexes and constraints ask the router, before they touch a table, whether that database is to have the model (Routing between databases: ConnectionRouter). They lean on what every field, index and constraint provides, deconstruct and clone, and on a manager’s deconstruct: the arguments an object would be made again with are what makes it comparable between two states and writable into a file (Models and fields). The state’s registry is an Apps like the main one (The registry and its models). How a command is found and run, and the system checks that run before migrate, are in Management commands. A test run gives its databases their tables by calling migrate (Test databases: BaseDatabaseCreation). And applications of django.contrib listen for post_migrate, to create permissions (Making the permissions: create_permissions) and content types (Making content types: create_contenttypes and RenameContentType).