12.1 A migration: its operations, its dependencies and how it is applied

The attributes a file sets

Migration gives each of its attributes a value, so the subclass that a migration file defines need state only those that differ. Migration.operations lists the operations in the order they run.

Migration.dependencies and Migration.run_before are lists of keys that the loader turns into edges of the graph, in opposite directions, though from a key whose name is "__first__" or "__latest__" it may make none (Finding migrations: MigrationLoader). A key in dependencies names a migration that must be applied before this one: its node becomes a parent of this one’s. A key in run_before names one that must be applied after: this migration’s node becomes a parent of that one’s, as if the other file had named this migration in its own dependencies (MigrationLoader.add_internal_dependencies, MigrationLoader.add_external_dependencies; The migration graph: MigrationGraph). The comment on run_before gives a use: to make third-party applications’ migrations run after a project’s replacement for the user model.

Migration.initial is None, True or False. Under migrate --fake-initial, MigrationExecutor.detect_soft_applied looks in the database for the tables and columns a migration would make, and where it finds them the executor records the migration as applied without running it. The method looks where initial is True, does not where it is False, and where it is None looks only if the migration depends on no migration of its own application (Applying a plan: MigrationExecutor). The autodetector sets initial on each migration it makes, to true where the application has no model in the state the migration files make, and the writer writes initial = True for a true value and nothing for any other (Writing a migration file: MigrationWriter).

Migration.atomic, true unless a file says otherwise, asks for the migration to run in one transaction. Migration.replaces lists the keys of the migrations this one stands in place of, which makes it a replacing migration (Squashed migrations: replaces and the squashmigrations command).

The instance: lists of its own, identity by key

Migration.__init__ keeps the name and the label, and gives the instance its own copy of each list of the class: the operations, the dependencies, Migration.run_before and Migration.replaces. Its comment gives the reason, that the lists may be changed at runtime, and they are: makemigrations --update extends the operations and the dependencies of a migration the loader made (Command.write_to_last_migration_files; The makemigrations command).

Two migrations are equal when their names and their labels are equal, whatever else they hold, and a migration’s hash is that of the string str makes of it, the label and the name with a dot between: migrations can be kept in a set and be the keys of a dict, as the executor keeps them.

mutate_state: the state halves alone

Migration.mutate_state runs the state half of each operation in order, on one project state, and returns that state. It touches no database, and is how the loader and the executor make a project state from migration files: MigrationGraph.make_state runs it for each migration of a forwards plan, and MigrationExecutor._create_project_state, called with with_applied_migrations=True, for each applied migration (Project state: ProjectState and its model states). Left to itself it clones the state and changes the clone; with preserve=False, which both of those pass, it changes the state it was given.

apply: both halves, forwards

Migration.apply is given the project state as the migrations before this one left it, and a schema editor. For each operation in turn it clones the state, runs the state half on the state it was given, and calls the database half forwards with the clone as the from-state and the state as the to-state. A database half may need both (Operations: the base class and its two halves). What apply returns is the object it was given, moved on by every operation, for the next migration to start from: MigrationExecutor.apply_migration, which calls apply inside a schema editor it has opened, returns it, and MigrationExecutor._migrate_all_forwards gives it to the apply_migration of the next migration.

Here yard’s first migration is applied by hand, as apply_migration applies it, to a state rendered as the executor renders it. In a recording a call is set under the call that made it, with the migration or operation it was called on in square brackets and what it returned after ->, and a line after a bar is a line of a file; only the lines a passage is about are written, and what ran between them is left out.

Migration.apply(project_state=state, schema_editor=the schema editor)  [yard.0001_initial]
  ProjectState.clone  ->  clone 1, a ProjectState of 11 model states, its registry rendered
  CreateModel.state_forwards(state=state)  [Create model Dock]
  CreateModel.database_forwards(from_state=clone 1, to_state=state)  [Create model Dock]
  ProjectState.clone  ->  clone 2, a ProjectState of 12 model states, its registry rendered
  CreateModel.state_forwards(state=state)  [Create model Repair]
  CreateModel.database_forwards(from_state=clone 2, to_state=state)  [Create model Repair]
  -> state

An atomic block for the migration, or for one operation

The schema editor that the executor makes with atomic=migration.atomic holds one atomic block open from its entry to its exit where that argument is true and the database can roll back DDL, the statements that change a schema, as SQLite and PostgreSQL can and MySQL and Oracle cannot; BaseDatabaseSchemaEditor.atomic_migration says whether it does (The schema editor: BaseDatabaseSchemaEditor). Migration.apply and Migration.unapply add a block only where the editor has none: an operation’s database half then runs in an atomic block of its own when Operation.atomic is true, or when it is anything but False and Migration.atomic is true.

Operation.atomic is False on the base class, whose comment says a true value is for forcing an operation that has no DDL, like RunPython, to be atomic even where DDL cannot be rolled back. RunPython is the one operation in Django that sets it otherwise, on each instance, to the argument it was made with: atomic=None unless one is given (RunPython, RunSQL and SeparateDatabaseAndState).

unapply: backwards, in two passes

Migration.unapply is given what Migration.apply is given, the state before the migration, and has to call each operation’s database half backwards, last operation first, each with the state after its operation as the from-state and the state before it as the to-state. Those states cannot be had by going backwards, since an operation has a state half forwards and none backwards: the state after the last operation exists only when every state half has run, in order, from the state before the first.

So unapply makes two passes. The first goes forwards and sends the database nothing. A state half works on the state it is given and makes no new one, so for each operation the pass clones the state so far and clones that clone, then runs the state half on the first of the two: the first is now the state after the operation, and the second is left as the state before it. It puts the operation and its two states at the front of a list, and the first clone is the state so far for the next operation. The second pass goes down the list, which has the last operation first, and calls each database half backwards with the states exchanged: the one reached after the operation is where the database stands, and the one before it is where it is to go. Atomic blocks are decided as in apply; the state given is not changed, and is returned.

The first pass begins each turn by reading Operation.reversible, and raises IrreversibleError, a RuntimeError, at the first operation for which it is false. Since the whole pass comes before any database half, such a migration is refused with none of its operations undone. reversible is true on Operation, and in Django only RunPython and RunSQL can have it false: theirs is a property, true only where the operation was given something to run backwards.

collect_sql=True: comments among the statements

Given collect_sql=True, Migration.apply and Migration.unapply write comments among the statements a collecting schema editor keeps as text. MigrationLoader.collect_sql calls them so, and sqlmigrate prints the result (Migration.apply, MigrationLoader.collect_sql; The migrate command, showmigrations and sqlmigrate). Before an operation’s database half the migration appends three comment lines to BaseDatabaseSchemaEditor.collected_sql, the middle one holding what Operation.describe returns. Where Operation.reduces_to_sql is false, which in Django is RunPython alone, it appends a fourth and goes on to the next operation: apply has then run neither half of that one, and unapply, whose first pass ran its state half, not its database half. After any other operation it appends "-- (no-op)" if the database half added nothing to the list. What sqlmigrate prints for a migration with a RunPython is in RunPython, RunSQL and SeparateDatabaseAndState.

suggest_name

Migration.suggest_name proposes the part of a migration’s name that follows its number. Where no --name was given, MigrationAutodetector.arrange_for_graph asks it for each new migration that is not its application’s first, and makemigrations --update for the one it rewrites (Migration.suggest_name; From operations to migrations: dependencies, order and names). Where Migration.initial is true the answer is "initial". Otherwise it takes each operation’s Operation.migration_name_fragment, with every run of characters other than letters, digits and underscores made an underscore. If there is no operation, or one of them has no fragment, as RunPython, RunSQL and SeparateDatabaseAndState have none, the answer is "auto_" and the date and the time to the minute, a fallback the docstring calls an effort to avoid conflicts in version control. Otherwise the name is the first fragment, whatever its length, with each later one joined on by an underscore until one would make the name longer than 52 characters; there "_and_more" is added instead, and the rest are left out. For yard’s second migration, of four operations, the answer was "rename_started_repair_begun_dock_metres_and_more": its third fragment, "alter_dock_name", would have made a name of 55 characters.

swappable_dependency and SwappableTuple

A migration may depend on whatever model a setting names, as one with a foreign key to the user model depends on AUTH_USER_MODEL. Its file then holds a call where the key would be, as in the migration that gives yard’s Repair such a foreign key, made with django.contrib.auth installed:

|     dependencies = [
|         ('yard', '0008_alter_repair_foreman'),
|         migrations.swappable_dependency(settings.AUTH_USER_MODEL),

swappable_dependency is given the setting’s value as the file is imported, and returns the label that value begins with, paired with "__first__", as a SwappableTuple: a tuple that also carries the value it was made from. To the loader it is a key like any other, whose "__first__" MigrationLoader.check_key resolves, here to ("auth", "0001_initial"), the first migration of auth (Finding migrations: MigrationLoader). Squashmigrations reads the value the tuple carries: where that is the current value of AUTH_USER_MODEL, the file it writes has the call again (Command.handle; Squashed migrations: replaces and the squashmigrations command).