12.19 Writing a migration file: MigrationWriter

A migration file, part by part

The text of a migration file is a fixed frame with slots, and MigrationWriter.as_string fills each, some with text made elsewhere.

How a value becomes an expression is in Serializing values: serializer_factory and deconstruct: MigrationWriter.serialize returns the serializer’s answer for a value, its text and the import lines the text needs, and MigrationWriter.register_serializer and MigrationWriter.unregister_serializer only call Serializer.register and Serializer.unregister.

MigrationWriter.as_string: the text

as_string fills the slots of MIGRATION_TEMPLATE, whose own text is the class line and the brackets of the two lists. For each operation of the migration it makes an OperationWriter, whose OperationWriter.serialize returns the operation’s text and its import lines.

A dependency is a key and is serialized as the tuple it is, with one exception. The autodetector gives the dependency that a foreign key to the user model makes the label "__setting__" and, in the place of a migration’s name, the name of the setting, AUTH_USER_MODEL (From operations to migrations: dependencies, order and names). For that label as_string writes a call of swappable_dependency on the setting and adds from django.conf import settings to the imports. The call is made at each import, so the file never fixes the application the setting names; it returns a SwappableTuple, the label of the setting’s model with "__first__", carrying the setting’s value. The dependencies’ lines are sorted as text.

In the recordings here, from a running Django, a line after a bar is a line of a text or a file and a line with -> a question and its answer; only the lines a passage is about are written. For a migration whose one dependency is ("__setting__", "AUTH_USER_MODEL"), a writer’s text has these lines:

| from django.conf import settings
...
|         migrations.swappable_dependency(settings.AUTH_USER_MODEL),

The label is all the writer goes by. A migration loaded from a file holds the SwappableTuple the call returned, and a writer given that migration writes a plain tuple, which fixes the application the setting named when the file was loaded. makemigrations had written the call in the file of yard’s ninth migration. As the loader made that migration, with django.contrib.auth installed, its dependencies have these types, and a writer’s text for it these lines:

[type(dependency).__name__ for dependency in migration.dependencies]  ->  ['tuple', 'SwappableTuple']
...
  |     dependencies = [
  |         ('auth', '__first__'),
  |         ('yard', '0008_alter_repair_foreman'),
  |     ]

Of a migration’s other attributes, Migration.replaces and Migration.run_before are written where they are not empty, Migration.initial where it is true and Migration.atomic where it is false; dependencies and operations always. The header, MIGRATION_HEADER_TEMPLATE, is one comment line with the answer of get_version and the time to the minute from django.utils.timezone.now, which is UTC where USE_TZ is true; a writer made with include_header=False, as under --no-header, writes none.

The imports, and functions to be copied by hand

The import lines are a set of strings, gathered as the operations and the dependencies were written, and MigrationWriter.as_string changes the set before it writes it.

First it takes out every import line in which a name after a dot begins with a digit: the test is for a migration module named as makemigrations names a new one, number first. A RunPython whose function is defined in such a module asks for such a line, since a function is serialized as its module and its name with an import of the module, and no import statement or dotted expression can name a module such as yard’s 0004_name_the_foremen. The writer does not copy the function’s source. It drops the line, sets MigrationWriter.needs_manual_porting for the command to read (Who makes a writer), and lists the modules in a comment under the imports; the operation’s text still names the function by its module, so the file cannot be imported until someone has done what the comment asks. The comment in the file squashmigrations wrote for yard, and its RunPython:

| # Functions from the following migrations need manual copying.
| # Move them and any dependencies into this file, then update the
| # RunPython operations to refer to the local versions:
| # harbour.yard.migrations.0004_name_the_foremen
...
|         migrations.RunPython(
|             code=harbour.yard.migrations.0004_name_the_foremen.name_foremen,
|             reverse_code=harbour.yard.migrations.0004_name_the_foremen.forget_foremen,
|         ),

Then from django.db import models, where a value asked for it, is replaced by from django.db import migrations, models; where none did, from django.db import migrations is added, which the class line needs. Last, the sort: import lines before from lines, each kind by the module it names.

OperationWriter: an operation as a call

An OperationWriter builds one operation’s text with OperationWriter.feed, which appends a line at the writer’s level of indentation (four spaces to a level; by default 2, inside the class and its list), and OperationWriter.render, which joins the lines. OperationWriter.serialize calls the operation’s Operation.deconstruct, which returns a name, positional arguments and keyword arguments (Operation.deconstruct; Operations: the base class and its two halves), and asks get_func_args for the names of the constructor’s parameters. Where django.db.migrations has an attribute of that name and it is the operation’s class, the call opens with migrations. and the name, leaning on the import every file has; otherwise with the class’s module and the name, and an import of the module is added.

Every argument is written by keyword and begins a line. A positional argument takes the name of the parameter in its position. Keyword arguments follow in the order of the constructor’s signature, and only those that are parameters of it: one that deconstruct returns under another name is left out, and nothing is raised.

How an argument is laid out depends on Operation.serialization_expand_args, the names of the arguments to spread over lines. One named there whose value is a list, a tuple or a dictionary is opened on a line, and each item, or each key with its value, is serialized on a line of its own, one level in; any other argument is serialized whole after its name. The base class’s list is empty, and these fill it: CreateModel ("fields", "options", "managers"), AlterModelManagers ("managers") and SeparateDatabaseAndState (its two lists). A value whose text has several lines is fed line by line at the writer’s level, which is how the operations inside a SeparateDatabaseAndState come out indented under it: each is written by an OperationWriter of its own that begins at level 0.

Where the file goes: MigrationWriter.basedir

MigrationWriter.filename is the migration’s name and .py, and MigrationWriter.path joins it to basedir, a property: the directory of the application’s migrations package, which may not exist yet.

It asks MigrationLoader.migrations_module for the package’s name: the application’s entry in MIGRATION_MODULES, or the application’s own package and "migrations" (Finding migrations: MigrationLoader). For an entry of None, an application with no migrations, basedir raises ValueError. A package that can be imported, and whose directory module_dir can tell, answers with it. Of the others, a package directly inside the application’s own, as the default one is, has its place under AppConfig.path, in a directory of its last name: basedir returns that path and makes nothing, and makemigrations makes the directory, with its __init__.py (Command.write_migration_files; The makemigrations command). What is left is a package that MIGRATION_MODULES puts elsewhere: basedir drops names from the right of the dotted name until what remains can be imported and has a directory, makes under it the directory of each name it dropped, writes an empty __init__.py into each, over one that is there, and returns the last, or raises ValueError where no part of it does both.

Who makes a writer

A MigrationWriter is made by makemigrations (The makemigrations command), by squashmigrations (Squashed migrations: replaces and the squashmigrations command) and by optimizemigration (The optimizer: MigrationOptimizer and reduce).