23.4 What changed in querysets

Fetch modes

Until 6.1, reading a deferred field, or a related object that an instance had not cached, ran a query for that one instance. Since 6.1 the read is handed to the instance’s fetch mode, an object that decides how the value is fetched. FETCH_ONE, the default, fetches it for the one instance, which the release notes call Django’s existing behaviour. FETCH_PEERS fetches it at once for every instance that the same run of the queryset made, the instance’s peers. FETCH_RAISE refuses, with FieldFetchBlocked, a FieldError. The notes give the new modes their purposes: the peers bring most cases of the “N+1 queries problem” down to two queries, and the refusal keeps unintended queries out of code where they would cost.

QuerySet.fetch_mode sets a mode on a clone of the queryset, which hands it to each instance made from a row through a keyword argument that 6.1 added to Model.from_db, fetch_mode=None. Deferred fields and fetch modes in a queryset tells the mechanism, and Reading and setting related objects what a relation’s descriptor does under each mode. Here four sailors, two to a ship, have their ships read in a loop:

# fetch_modes.py at this commit: [s.ship.name for s in Sailor.objects.fetch_mode(mode).order_by("id")]
FETCH_ONE     5 statements   ['Petrel', 'Petrel', 'Gannet', 'Gannet']
FETCH_PEERS   2 statements   ['Petrel', 'Petrel', 'Gannet', 'Gannet']
FETCH_RAISE   1 statement    FieldFetchBlocked: Fetching of Sailor.ship blocked.

Under the default each sailor’s ship was fetched for that sailor, Petrel twice; under the peers the first read fetched the four sailors’ ships, by a prefetch of one statement.

In 6.1 a queryset of a model whose own from_db accepts neither fetch_mode= nor arbitrary keywords crashed when iterated; 6.1.1 made such overrides work again, and deprecated them. At this commit _get_from_db inspects the override, warns with RemovedInDjango2028Warning, and calls it without the mode, so its instances are not given the queryset’s.

Beside fetch modes, 6.1 deprecated select_related with no arguments, which has the compiler follow the relations that cannot be null. The call warns with RemovedInDjango2028Warning, the notes name FETCH_PEERS as the other way to the related objects, and the deprecation timeline has the call raise TypeError in Django 2028. What still warns: the deprecations and the releases that end them lists both deprecations.

The notes of 6.1.2, a release expected and not yet made at this commit, list as fixed a bug of 6.1 in which FETCH_PEERS did not batch the instances that select_related loaded. At this commit each RelatedPopulator, which makes the instances of one joined model, keeps a list of peers of its own under a mode that tracks them, so the ships joined to one run of sailors are each other’s peers.

Orderings

QuerySet.first and QuerySet.last give a queryset with no ordering one by primary key. Until 6.1 they did so also for a queryset whose ordering had been cleared on purpose, by QuerySet.order_by with no arguments. Since 6.1 they read the mark that call leaves, Query.default_ordering turned off, and use the queryset as it stands (QuerySet.first; get, first, count and the other methods that run a query at once).

A queryset made by QuerySet.union, QuerySet.intersection or QuerySet.difference has the model’s default ordering since 6.1: QuerySet._combinator_query clears the ordering of the combined query and turns its default back on. The notes give the consequence: where values or values_list has not selected a field of that ordering, the statement fails with DatabaseError, and order_by() with no arguments on the combined queryset clears the default. Ship is ordered by name:

# combined.py at this commit, on SQLite; left and right are each Ship.objects.order_by().values("tonnage")
list(left.union(right))              DatabaseError: ORDER BY term does not match any column in the result set.
list(left.union(right).order_by())   [{'tonnage': 480}, {'tonnage': 1200}]

The parts were cleared first: each keeps a default ordering of its own, which order_by() on the combined queryset leaves in place, and a backend that cannot order the parts of a compound statement, as SQLite cannot, refuses that with another DatabaseError (SQLCompiler.get_combinator_sql; Combining querysets: the operators and union).

QuerySet.totally_ordered (6.1) is a property beside QuerySet.ordered, true only where the ordering is deterministic (QuerySet.totally_ordered; Annotations, ordering and the other methods that change the query).

Since 5.2 the columns of the statement of QuerySet.values and QuerySet.values_list stand in the order the names were given; the notes say the order had followed counterintuitive rules that made combining such querysets with union unpredictable. Query.set_values records each name, with where its column comes from, in Query.selected, and SQLCompiler.get_select writes the select list from it. Given no names, values leaves selected as None, and the compiler’s own order stands: the extra columns, the model’s fields, the annotations (values and values_list: rows as dictionaries and tuples).

The smaller changes