12.20 The makemigrations command

The loader, and each database’s history

Command.handle makes its loader as MigrationLoader(None, ignore_no_migrations=True). With no connection the loader asks no database what has been applied and counts nothing as applied. A comment gives the purpose, that replaced migrations are not to be resolved from a database: with none applied, every replacing migration (a squashed one) that no other replaces stands in the graph in place of those it replaces, whatever a developer’s database has had (Squashed migrations: replaces and the squashmigrations command). ignore_no_migrations=True quiets two errors of Finding migrations: MigrationLoader: an application for which MIGRATION_MODULES names a module that is not there is counted as unmigrated, and a dependency on "__first__" or "__latest__" of a migrated application with no migration yet is dropped.

The history is checked all the same, on real connections. The aliases are all of DATABASES where DATABASE_ROUTERS is not empty and "default" alone where it is, in sorted order. One is passed over if its engine is the dummy backend, which a project with no database configured has, or if the router lets no model of any installed application be migrated there. On the others MigrationLoader.check_consistent_history reads the database’s table and raises InconsistentMigrationHistory, which ends the command, if a migration applied there, and in the loader’s graph, has a parent, a migration it depends on, that is not applied there. An unapplied parent is let pass where it is a replacing migration each of whose replaced migrations is applied or is, in turn, such a replacing migration. An OperationalError, as from a database that cannot be reached, becomes a RuntimeWarning, and the command goes on.

Two leaf nodes, and --merge

For makemigrations, MigrationLoader.detect_conflicts gives, for each application with more than one leaf node, those nodes’ names; where labels were given on its command line, the command drops other applications’ conflicts. If any is left and --merge was not given, the command stops with a CommandError that names them: the docstring of MigrationGraph.leaf_nodes calls more than one leaf in an application an error, usually the result of a merge in version control, that needs the user. With --merge and none left, it logs that there is nothing to merge and returns.

With --merge, Command.handle_merge takes each application in conflict in turn. For each leaf it works out an ancestry within the application: the nodes of MigrationGraph.forwards_plan for the leaf, less those of other applications. It reads the ancestries side by side from the start and counts how far they hold the same node: those are the common ancestors, and with none it raises ValueError. What follows them in a leaf’s ancestry is its branch, and the operations of the branch’s migrations, in order, are printed under the leaf’s name at verbosity 1 or more.

Nothing tests whether the branches can be merged: InteractiveMigrationQuestioner.ask_merge puts the question to the user, and its text says what the answer rests on; with --noinput the questioner is a plain MigrationQuestioner whose default for "ask_merge" is yes, and nothing is asked (The questioner: MigrationQuestioner).

In harbour, yard has two migrations numbered 0006, yard.0006_dock_covered and yard.0006_long_dock, both depending on yard.0005_repair_by_foreman, so each branch is its leaf alone. In the recordings here a line beginning python manage.py is a command line; under it stdout | or stderr | begins a line the command printed on that stream, stdin | a line typed to it and a bar a line of a file it wrote; only the lines a passage is about are quoted.

python manage.py makemigrations --merge --name merge
  stdout | Merging yard
  stdout |   Branch 0006_dock_covered
  stdout |     + Add field covered to dock
  stdout |   Branch 0006_long_dock
  stdout |     s Raw SQL operation
  stdout |
  stdout | Merging will only work if the operations printed above do not conflict
  stdout | with each other (working on different fields or models)
  stdout | Should these migration branches be merged? [y/N]
  stdin  | y
  stdout |
  stdout | Created new merge migration harbour/yard/migrations/0007_merge.py

On a yes the command makes the migration itself. Its number is one more than the largest that MigrationAutodetector.parse_number reads from the leaves’ names, or 2 where it reads none. After the number comes the value of --name, or else "merge" followed by the leaves’ names, sorted and joined with underscores, or by the date and the time where the joined names are longer than 47 characters. Unless this is a dry run, a MigrationWriter’s text for it is written to the writer’s path. Having no operations, the migration changes no state and no database: it gives the application one leaf node again.

The two states and changes

Without --merge, Command.handle next chooses a questioner: an InteractiveMigrationQuestioner unless --noinput was given, and then a NonInteractiveMigrationQuestioner. Each is given the labels and whether this is a dry run. The autodetector is made of the from-state, MigrationLoader.project_state for the whole graph, the to-state, ProjectState.from_apps of the main registry, and the questioner.

MigrationAutodetector.changes is called with the loader’s graph, the value of --name, and the labels twice, or None twice where there were none. As the applications to trim to, the labels cut the result down to those applications and the ones their new migrations depend on. What returns is a dictionary from a label to that application’s new migrations, numbered, named and given their dependencies (From operations to migrations: dependencies, order and names). Where it is empty the command logs, at verbosity 1 or more, that no changes were detected, and returns.

As the applications to convert, the labels matter to an application with no migrations package: a plain makemigrations makes nothing for it, and one given its label makes its first migration. The loader counts such an application as unmigrated, and the from-state is given the unmigrated applications as its ProjectState.real_apps: the autodetector leaves their models out of the comparison unless the label is one of those to convert (The autodetector: comparing two project states). A graph with no leaf node gives a from-state with no real_apps, so in a project with no migration at all every application’s models are compared. MigrationAutodetector.arrange_for_graph then asks MigrationQuestioner.ask_initial of each application that has new migrations and no leaf node, and drops them on a no: yes for a label that was given or a package whose only Python file is "__init__.py", no for an application with no package (The questioner: MigrationQuestioner). So where no application has a migrations package yet, a plain makemigrations finds no changes.

--empty leaves the comparison out. It requires a label, and for each it makes a Migration with no operations, puts them in a dictionary of the shape changes returns, and hands that to MigrationAutodetector.arrange_for_graph for a number, a name and a dependency on the application’s leaf node, where it has one. The files are written like any others.

--check is --dry-run with a verdict on the comparison: where changes returns migrations, the command exits with status 1 after its dry run of them; where it returns none the status is 0. --empty and --merge return before that exit.

Writing the files: write_migration_files

Command.write_migration_files takes a dictionary from a label to that application’s migrations, an application at a time. For each migration it makes a MigrationWriter, one that writes no header comment under --no-header, and, at verbosity 1 or more, logs the file’s path and a line for each operation: Operation.formatted_description, the sign of its Operation.category, or "?" where it has none, and what its Operation.describe returns. Unless this is a dry run it then makes the path’s directory and an empty "__init__.py" there if either is missing, has the writer make the text, and writes it to the path (Writing a migration file: MigrationWriter). At the end, as after a merge, run_formatters runs the formatter black over the files written, if a program of that name is on the path.

In a dry run at verbosity 3 each file’s text is logged under a heading that names it. --scriptable sends everything the command logs, with the questioner’s prompts, to standard error, and each file’s path to standard output: Command.log, through which the logged lines are written, takes its stream from Command.log_output.

python manage.py makemigrations yard --scriptable
  stdout | harbour/yard/migrations/0006_dock_covered.py
  stderr | Migrations for 'yard':
  stderr |   harbour/yard/migrations/0006_dock_covered.py
  stderr |     + Add field covered to dock

--update: the leaf migration written again

With --update the changes are folded into each application’s leaf migration by Command.write_to_last_migration_files. Here yard.0003_repair_foreman, which adds Repair.foreman with a default typed in answer to the questioner, was written and not yet applied, and the field was then made longer in the models:

python manage.py makemigrations yard --update
  stdout | Migrations for 'yard':
  stdout |   harbour/yard/migrations/0003_repair_foreman_updated.py
  stdout |     + Add field foreman to repair
  stdout | Deleted harbour/yard/migrations/0003_repair_foreman.py
...
|     operations = [
|         migrations.AddField(
|             model_name='repair',
|             name='foreman',
|             field=models.CharField(max_length=80),
|             preserve_default=False,
|         ),
|     ]

The method makes a loader of its own, with the connection "default": the first loader counts nothing as applied, and one of the refusals turns on what is. For each application with changes it takes the leaf node of that loader’s graph and raises CommandError if there is none, if the leaf is a replacing migration, if it is applied on "default" (no other database is asked), or if a migration on disk names it among its dependencies.

It then appends to the leaf’s operations those of each new migration, and to its dependencies those of theirs whose label is not the application’s own: another application’s migration, or ("__setting__", "AUTH_USER_MODEL") for a relation to the user model. The whole list of operations is put through MigrationOptimizer.optimize (The optimizer: MigrationOptimizer and reduce). In the recording the optimizer made one AddField of the old AddField and the new AlterField, and the default that was typed is not in it: applied, the migration has no value for the rows the table has, and on SQLite it fails if there is one.

The new name is the first four characters of the old, an underscore, and the value of --name or else Migration.suggest_name of the rewritten migration; where that is the old name again, "_updated" is added to it, so the name always changes, and the new file is never the one about to be removed. Command.write_migration_files is then given the rewritten migrations and each old file’s path. When a new file has been written the old one is removed, unless the writer set MigrationWriter.needs_manual_porting, having met a function that must be copied by hand out of a migration module: then the old file is kept, and a warning says so.