12.18 Serializing values: serializer_factory and deconstruct

serializer_factory: the order of the tests

serializer_factory is a fixed sequence of tests, and the first a value passes decides. It first unwraps: a Promise, which a lazily translated string is, becomes its str then, and a LazyObject the object it wraps.

Then a field gets ModelFieldSerializer, a manager ModelManagerSerializer, an operation OperationSerializer, a class TypeSerializer, and anything else with an attribute deconstruct gets DeconstructibleSerializer. A class whose instances can deconstruct has the attribute too, and is caught by the test before.

What passes none of these reaches the registry, Serializer._registry: a dictionary from a type, or a tuple of types, to a serializer class. Its entries are tried in the dictionary’s order with isinstance, and some depend on that order: models.Choices stands before enum.Enum and datetime.datetime before datetime.date, each a subclass of what it precedes; SettingsReference before str; pathlib.PurePath before os.PathLike; and str, bytes, range and the containers before collections.abc.Iterable, which each of them is.

type(serializer_factory(Rank.MATE)).__name__  ->  'ChoicesSerializer'
type(serializer_factory(uuid.SafeUUID.safe)).__name__  ->  'EnumSerializer'

Rank is harbour’s class of text choices, uuid.SafeUUID an enumeration of Python’s. SET returns a function, which the entry for functions would take, but sets a deconstruct on it, and the attribute is tested first.

deconstruct: a value written as a call

The deconstruct protocol is how an object says how to make it again: the method returns the dotted path of something to call, positional arguments and keyword arguments, and DeconstructibleSerializer.serialize_deconstructed writes the call. A path whose module is exactly django.db.models is written models. and the name, with "from django.db import models"; any other whole, with an import of its module. The positional arguments go through serializer_factory in turn, then the keyword arguments sorted by name; those whose names are not identifiers go into a dictionary written after **. So an object can be written only if everything it deconstructs to can.

A field, a manager and an operation answer in other shapes, so each has its own serializer. Field.deconstruct returns the field’s name first, which ModelFieldSerializer leaves out, and shortens the path of a class under django.db.models.fields to django.db.models (Field.deconstruct; A field: the base class). ModelManagerSerializer writes a manager that QuerySet.as_manager made as a call of that method on the queryset class’s path, and another as a call of its own class (Managers). OperationSerializer hands an operation to OperationWriter (Writing a migration file: MigrationWriter).

Q.deconstruct, Index.deconstruct and BaseConstraint.deconstruct are written by hand, and each shortens its own module to django.db.models (Q.deconstruct; Constraints and indexes). Other classes get the method from the decorator deconstructible. It replaces the class’s __new__ with one that keeps the arguments of the call on the new object, as _constructor_args, and adds a deconstruct that returns them after a path. The path is the module and name of the object’s own class, so a subclass is written under its own name; an object of the decorated class itself gets the path passed as path=, where one was. The method first imports the module, and raises ValueError if the name is not in it, as an inner class’s is not. Tide is a class decorated with path="__main__.Tide":

written(Tide(high=4.2))  ->  __main__.Tide(high=4.2)   needs ['import __main__']
Tide(high=4.2).deconstruct()  ->  ('__main__.Tide', (), {'high': 4.2})

Expression is decorated, as is every validator class of django/core/validators.py; F, Func, Value and some others are decorated with a path in django.db.models, which is why a file has models.F and not django.db.models.expressions.F.

Values written by name

What is written by name is not written out: the file imports whatever has that name when the migration is loaded.

FunctionTypeSerializer takes what FUNCTION_TYPES lists, functions, built-in functions and methods among them. A method whose __self__ is a class is written as the class’s module, its qualified name and the method’s name. Anything else is written as its __module__ and __qualname__: a function at the top of a module, or in the body of a class. A method defined in Python and bound to an instance is written so too, as the function it wraps: the loaded migration has the function and no instance.

TypeSerializer writes a class by module and qualified name too, except models.Model and types.NoneType, written under those names, and a builtin, written bare with no import. EnumSerializer writes a member as its class and, in square brackets, its name, and a combined enum.Flag as its members joined by |. ChoicesSerializer writes a member of a Choices class not by name but as its value alone, so the file does not import the class.

A SettingsReference is a str that also holds a setting’s name, in SettingsReference.setting_name. A field makes one: where RelatedField.swappable_setting names a setting, as it does for a field that points at the user model, which AUTH_USER_MODEL can swap, ForeignObject.deconstruct and ManyToManyField.deconstruct put a SettingsReference in place of the "to" argument. It is written as the setting, so the file follows the setting of the project that loads it; the setting’s value by itself is written as a string:

written(SettingsReference(settings.AUTH_USER_MODEL, 'AUTH_USER_MODEL'))  ->  settings.AUTH_USER_MODEL   needs ['from django.conf import settings']
written(settings.AUTH_USER_MODEL)  ->  'auth.User'   needs no import

The dependency that goes with it is in From operations to migrations: dependencies, order and names.

Values written whole

Where a value’s repr is an expression that makes it, the repr is the text; BaseSimpleSerializer returns it with no import. The other serializers of this kind mend what repr leaves undone: an import it needs, the module’s name in front of it, or a call of float on a string for a float that is not a number or is infinite. DatetimeDatetimeSerializer first converts a datetime in a time zone other than UTC to UTC. PathSerializer writes a concrete path as the pure path of its flavour, so that, its comment says, a migration made on one platform can be used on another. RegexSerializer writes a pattern’s flags after an exclusive-or with those of an empty str pattern, and none where that leaves none: '^m-' compiled with re.IGNORECASE is written re.compile('^m-', 2), without the re.UNICODE that Python added.

For a list, a tuple or a set, BaseSequenceSerializer.serialize writes each item and joins the strings, and a subclass gives the brackets through BaseSequenceSerializer._format: a tuple of one item gets its comma, and an empty set is set(), since {} is a dictionary. BaseUnorderedSequenceSerializer first sorts a set or a frozenset by the repr of its items, and DictionarySerializer a dictionary by its items themselves, so the text does not depend on the order of iteration. IterableSerializer writes any other iterable, a generator too, as a tuple of what it yields.

The registry, in its order

The entries of Serializer._registry stand in the order serializer_factory tries them, so a row takes only what no row above it took.

What cannot be written

A function is written by name, and a lambda has none to import by: FunctionTypeSerializer refuses it, a function with no module, and one for which qualname raises, as it does for a function defined inside another, whose qualified name has "<" in it. What passes no test of serializer_factory gets the general ValueError. datetime.UTC by itself does: the registry has no key for a datetime.timezone, and inside a datetime it is only part of a repr. So does object() in these lines, in which nested is a function of the recording’s own that returns one defined inside it:

written(lambda: 1)  ->  raises ValueError: Cannot serialize function: lambda
written(nested())  ->  raises ValueError: Could not find function inner in __main__.\n
written(object())  ->  raises ValueError: Cannot serialize: <object object>\nThere are some values Django cannot serialize into migration files.\nFor more, see https://docs.djangoproject.com/en/dev/topics/migrations/#migration-serializing

Registering a serializer

Serializer.register takes a type, or a tuple of types, and a serializer class; it raises ValueError for a class that is no subclass of BaseSerializer and otherwise stores the pair, and Serializer.unregister pops a key; MigrationWriter.register_serializer and MigrationWriter.unregister_serializer only call the two. A new key goes to the end of the dictionary, so it is reached only by a value no entry before it matches; a key already there keeps its place and gets the new class. And the registry is reached only past the tests before it: a serializer registered for a field class, or for a class with a deconstruct, is never asked. Fathoms is a class with no deconstruct, and this is a serializer for it, which writes the class as if it stood in a module depths that harbour does not have:

class FathomsSerializer(BaseSerializer):
    def serialize(self):
        return f"harbour.depths.Fathoms({self.value.number})", {"import harbour.depths"}

MigrationWriter.register_serializer(Fathoms, FathomsSerializer)
list(Serializer._registry)[-1] is Fathoms  ->  True
written(Fathoms(12))  ->  harbour.depths.Fathoms(12)   needs ['import harbour.depths']