API Reference

models

class computedfields.models.ComputedFieldsModelType

Bases: django.db.models.base.ModelBase

Metaclass for computed field models.

Handles the creation of the db fields. Also holds the needed data for graph calculations and dependency resolving.

After startup the method _resolve_dependencies gets called by app.ready to build the dependency resolving map. To avoid the expensive calculations in production mode the map can be pickled into a map file by setting COMPUTEDFIELDS_MAP in settings.py to a writable file path and calling the management command createmap.

Note

The map file will not be updated automatically and therefore must be recreated by calling the management command createmap after model changes.

classmethod preupdate_dependent(instance, model=None, update_fields=None)

Create a mapping of currently associated computed fields records, that would turn dirty by a follow-up bulk action.

Feed the mapping back to update_dependent as old argument after your bulk action to update deassociated computed field records as well.

classmethod preupdate_dependent_multi(instances)

Same as preupdate_dependent, but for multiple bulk actions at once.

After done with the bulk actions, feed the mapping back to update_dependent_multi as old argument to update deassociated computed field records as well.

classmethod update_dependent(instance, model=None, update_fields=None, old=None)

Updates all dependent computed fields model objects.

This is needed if you have computed fields that depend on a model changed by bulk actions. Simply call this function after the update with the queryset containing the changed objects. The queryset may not be finalized by distinct or any other means.

>>> Entry.objects.filter(pub_date__year=2010).update(comments_on=False)
>>> update_dependent(Entry.objects.filter(pub_date__year=2010))

This can also be used with bulk_create. Since bulk_create returns the objects in a python container, you have to create the queryset yourself, e.g. with pks:

>>> objs = Entry.objects.bulk_create([
...     Entry(headline='This is a test'),
...     Entry(headline='This is only a test'),
... ])
>>> pks = set(obj.pk for obj in objs)
>>> update_dependent(Entry.objects.filter(pk__in=pks))

Note

This function cannot be used to update computed fields on a computed fields model itself. For computed fields models always use save on the model objects. You still can use update or bulk_create but have to call save afterwards:

>>> objs = SomeComputedFieldsModel.objects.bulk_create([
...     SomeComputedFieldsModel(headline='This is a test'),
...     SomeComputedFieldsModel(headline='This is only a test'),
... ])
>>> for obj in objs:
...     obj.save()

(This behavior might change with future versions.)

Special care is needed, if a bulk action contains foreign key changes, that are part of a computed field dependency chain. To correctly handle that case, provide the result of preupdate_dependent as old argument like this:

>>> # given: some computed fields model depends somehow on Entry.fk_field
>>> old_relations = preupdate_dependent(Entry.objects.filter(pub_date__year=2010))
>>> Entry.objects.filter(pub_date__year=2010).update(fk_field=new_related_obj)
>>> update_dependent(Entry.objects.filter(pub_date__year=2010), old=old_relations)

For completeness - instance can also be a single model instance. Since calling save on a model instance will trigger this function by the post_save signal it should not be invoked for single model instances, if they get saved anyway.

classmethod update_dependent_multi(instances, old=None)

Updates all dependent computed fields model objects for multiple instances.

This function avoids redundant updates if consecutive update_dependent have intersections, example:

>>> update_dependent(Foo.objects.filter(i='x'))  # updates A, B, C
>>> update_dependent(Bar.objects.filter(j='y'))  # updates B, C, D
>>> update_dependent(Baz.objects.filter(k='z'))  # updates C, D, E

In the example the models B and D would be queried twice, C even three times. It gets even worse if the queries contain record intersections, those items would be queried and saved several times.

The updates above can be rewritten as:

>>> update_dependent_multi([
...     Foo.objects.filter(i='x'),
...     Bar.objects.filter(j='y'),
...     Baz.objects.filter(k='z')])

where all dependent model objects get queried and saved only once. The underlying querysets are expanded accordingly.

Note

instances can also contain model instances. Don’t use this function for model instances of the same type, instead aggregate those to querysets and use update_dependent (as shown for bulk_create above).

Again special care is needed, if the bulk actions involve foreign key changes, that are part of computed field dependency chains. Use preupdate_dependent_multi to create a record mapping of the current state and after your bulk changes feed it back as old argument to this function.

computedfields.models.get_contributing_fks()

Get a mapping of models and their local fk fields, that are part of a computed fields dependency chain.

Whenever a bulk action changes one of the fields listed here, you have to create a listing of the currently associated records with preupdate_dependent and, after doing the bulk change, feed the listing back to update_dependent.

This mapping can also be inspected as admin view, if COMPUTEDFIELDS_ADMIN is set to True.

class computedfields.models.ComputedFieldsModel(*args, **kwargs)

Bases: django.db.models.base.Model

Base class for a computed fields model.

To use computed fields derive your model from this class and use the @computed decorator:

from django.db import models
from computedfields.models import ComputedFieldsModel, computed

class Person(ComputedFieldsModel):
    forename = models.CharField(max_length=32)
    surname = models.CharField(max_length=32)

    @computed(models.CharField(max_length=32))
    def combined(self):
        return u'%s, %s' % (self.surname, self.forename)

combined will be turned into a real database field and can be accessed and searched like any other field. During saving the value gets calculated and written to the database. With the method compute('fieldname') you can inspect the value that will be written, which is useful if you have pending changes:

>>> person = Person(forename='Leeroy', surname='Jenkins')
>>> person.combined             # empty since not saved yet
>>> person.compute('combined')  # outputs 'Jenkins, Leeroy'
>>> person.save()
>>> person.combined             # outputs 'Jenkins, Leeroy'
>>> Person.objects.filter(combined__<some condition>)  # used in a queryset

compute(fieldname)

Returns the computed field value for fieldname.

save(*args, **kwargs)

Saves the current instance. Override this in a subclass if you want to control the saving process.

The ‘force_insert’ and ‘force_update’ parameters can be used to insist that the “save” must be an SQL insert or update (or equivalent for non-SQL backends), respectively. Normally, they should not be set.

computedfields.models.computed(field, **kwargs)

Decorator to create computed fields.

field should be a model field suitable to hold the result of the decorated method. The decorator understands an optional keyword argument depends to indicate dependencies to related model fields. Listed dependencies will automatically update the computed field.

Examples:

• create a char field with no outer dependencies

  @computed(models.CharField(max_length=32))
  def ...
  

• create a char field with one dependency to the field name of a foreign key relation fk

  @computed(models.CharField(max_length=32), depends=['fk#name'])
  def ...
  

The dependency string is in the form 'rel_a.rel_b#fieldname', where the computed field gets a value from a field fieldname, which is accessible through the relations rel_a –> rel_b. A relation can be any of foreign key, m2m, o2o and their corresponding back relations.

Caution

With the dependency auto resolver you can easily create recursive dependencies by accident. Imagine the following:

class A(ComputedFieldsModel):
    @computed(models.CharField(max_length=32), depends=['b_set#comp'])
    def comp(self):
        return ''.join(b.comp for b in self.b_set.all())

class B(ComputedFieldsModel):
    a = models.ForeignKey(A)

    @computed(models.CharField(max_length=32), depends=['a#comp'])
    def comp(self):
        return a.comp

Neither an object of A or B can be saved, since the comp fields depend on each other. While it is quite easy to spot for this simple case it might get tricky for more complicated dependencies. Therefore the dependency resolver tries to detect cyclic dependencies and raises a CycleNodeException if a cycle was found.

If you experience this in your project try to get in-depth cycle information, either by using the rendergraph management command or by accessing the graph object directly under your_model._graph. Also see the graph documentation here.

class computedfields.models.ComputedFieldsAdminModel(*args, **kwargs)

Bases: django.contrib.contenttypes.models.ContentType

Proxy model to list all ComputedFieldsModel models with their field dependencies in admin. This might be useful during development. To enable it, set COMPUTEDFIELDS_ADMIN in settings.py to True.

exception DoesNotExist

Bases: django.contrib.contenttypes.models.DoesNotExist

exception MultipleObjectsReturned

Bases: django.contrib.contenttypes.models.MultipleObjectsReturned

class computedfields.models.ContributingModelsModel(*args, **kwargs)

Bases: django.contrib.contenttypes.models.ContentType

Proxy model to list all models in admin, that contain fk fields contributing to computed fields. This might be useful during development. To enable it, set COMPUTEDFIELDS_ADMIN in settings.py to True. An fk field is considered contributing, if it is part of a computed field dependency, thus a change to it would impact a computed field.

exception DoesNotExist

Bases: django.contrib.contenttypes.models.DoesNotExist

exception MultipleObjectsReturned

Bases: django.contrib.contenttypes.models.MultipleObjectsReturned

admin

class computedfields.admin.ComputedModelsAdmin(model, admin_site)

Bases: django.contrib.admin.options.ModelAdmin

Shows all ComputedFieldsModel models with their field dependencies in the admin. Also renders the dependency graph if the graphviz package is installed.

has_add_permission(request, obj=None)

Returns True if the given request has permission to add an object. Can be overridden by the user in subclasses.

has_delete_permission(request, obj=None)

Returns True if the given request has permission to change the given Django model instance, the default implementation doesn’t examine the obj parameter.

Can be overridden by the user in subclasses. In such case it should return True if the given request has permission to delete the obj model instance. If obj is None, this should return True if the given request has permission to delete any object of the given type.

class computedfields.admin.ContributingModelsAdmin(model, admin_site)

Bases: django.contrib.admin.options.ModelAdmin

Shows models with cf contributing local fk fields.

has_add_permission(request, obj=None)

Returns True if the given request has permission to add an object. Can be overridden by the user in subclasses.

has_delete_permission(request, obj=None)

Returns True if the given request has permission to change the given Django model instance, the default implementation doesn’t examine the obj parameter.

Can be overridden by the user in subclasses. In such case it should return True if the given request has permission to delete the obj model instance. If obj is None, this should return True if the given request has permission to delete any object of the given type.

graph

Module containing the graph logic for the dependency resolver.

Upon application initialization a dependency graph of all project wide computed fields is created. The graph does a basic cycle check and removes redundant dependencies. Finally the dependencies are translated to the resolver map to be used later by update_dependent and in the signal handlers.

exception computedfields.graph.CycleException

Bases: Exception

Exception raised during path linearization, if a cycle was found. Contains the found cycle either as list of edges or nodes in args.

exception computedfields.graph.CycleEdgeException

Bases: computedfields.graph.CycleException

Exception raised during path linearization, if a cycle was found. Contains the found cycle as list of edges in args.

exception computedfields.graph.CycleNodeException

Bases: computedfields.graph.CycleException

Exception raised during path linearization, if a cycle was found. Contains the found cycle as list of nodes in args.

class computedfields.graph.Edge(left, right, data=None)

Bases: object

Class for representing an edge in Graph. The instances are created as singletons, calling Edge('A', 'B') multiple times will always point to the same object.

class computedfields.graph.Node(data)

Bases: object

Class for representing a node in Graph. The instances are created as singletons, calling Node('A') multiple times will always point to the same object.

class computedfields.graph.Graph

Bases: object

Simple directed graph implementation.

add_node(node)

Add a node to the graph.

remove_node(node)

Remove a node from the graph.

Warning

Removing edges containing the removed node is not implemented.

add_edge(edge)

Add an edge to the graph. Automatically inserts the associated nodes.

remove_edge(edge)

Removes an edge from the graph.

Warning

Does not remove leftover contained nodes.

get_dot(format='pdf', mark_edges=None, mark_nodes=None)

Returns the graphviz object of the graph (needs the graphviz package).

render(filename=None, format='pdf', mark_edges=None, mark_nodes=None)

Renders the graph to file (needs the graphviz package).

view(format='pdf', mark_edges=None, mark_nodes=None)

Directly opens the graph in the associated desktop viewer (needs the graphviz package).

edgepath_to_nodepath(path)

Converts a list of edges to a list of nodes.

nodepath_to_edgepath(path)

Converts a list of nodes to a list of edges.

get_edgepaths()

Returns a list of all edge paths. An edge path is represented as list of edges.

Might raise a CycleEdgeException. For in-depth cycle detection use edge_cycles, node_cycles` or get_cycles().

get_nodepaths()

Returns a list of all node paths. A node path is represented as list of nodes.

Might raise a CycleNodeException. For in-depth cycle detection use edge_cycles, node_cycles or get_cycles().

get_cycles()

Gets all cycles in graph.

This is not optimised by any means, it simply walks the whole graph recursively and aborts as soon a seen edge gets entered again. Therefore use this and all dependent properties (edge_cycles and node_cycles) for in-depth cycle inspection only.

As a start node any node on the left side of an edge will be tested.

Returns a mapping of

{frozenset(<cycle edges>): {
    'entries': set(edges leading to the cycle),
    'path': list(cycle edges in last seen order)
}}

An edge in entries is not necessarily part of the cycle itself, but once entered it will lead to the cycle.

edge_cycles

Returns all cycles as list of edge lists. Use this only for in-depth cycle inspection.

node_cycles

Returns all cycles as list of node lists. Use this only for in-depth cycle inspection.

is_cyclefree

True if the graph contains no cycles.

For faster calculation this property relies on path linearization instead of the more expensive full cycle detection. For in-depth cycle inspection use edge_cycles or node_cycles instead.

remove_redundant()

Find and remove redundant edges. An edge is redundant if there there are multiple possibilities to reach an end node from a start node. Since the longer path triggers more needed database updates the shorter path gets discarded. Might raise a CycleNodeException.

Returns the removed edges.

class computedfields.graph.ComputedModelsGraph(computed_models)

Bases: computedfields.graph.Graph

Class to convert the computed fields dependency strings into a graph and generate the final resolver functions.

Steps taken:

• resolve_dependencies resolves the depends field strings to real model fields.
• The dependencies are rearranged to adjacency lists for the underlying graph.
• The graph does a cycle check and removes redundant edges to lower the database penalty.
• In generate_lookup_map the path segments of remaining edges are collected into the final lookup map.

resolve_dependencies(computed_models)

Converts all depend strings into real model field lookups.

generate_lookup_map()

Generates the final lookup map for queryset generation.

Structure of the map is:

{model: {
    '#'      :  dependencies
    'fieldA' :  dependencies
    }
}

model denotes the source model of a given instance. 'fieldA' points to a field that was changed. The right side contains the dependencies in the form

{dependent_model: (fields, filter_strings)}

In update_dependent the information will be used to create a queryset and save their elements (roughly):

queryset = dependent_model.objects.filter(string1=instance)
queryset |= dependent_model.objects.filter(string2=instance)
for obj in queryset:
    obj.save(update_fields=fields)

The '#' is a special placeholder to indicate, that a model object was saved normally. It contains the plain and non computed field dependencies.

The separation of dependencies to computed fields and to other fields makes it possible to create complex computed field dependencies, even multiple times between the same objects without running into circular dependencies:

class A(ComputedFieldsModel):
    @computed(..., depends=['b_set#comp_b'])
    def comp_a(self):
         ...

class B(ComputedFieldsModel):
    a = ForeignKey(B)
    @computed(..., depends=['a'])
    def comp_b(self):
        ...

Here A.comp_a depends on b.com_b which itself somehow depends on A. If an instance of A gets saved, the corresponding objects in B will be updated, which triggers a final update of comp_a fields on associated A objects.

Caution

If there are only computed fields in update_fields always use those dependencies, never '#'. This is important to ensure cycle free database updates. For computed fields the corresponding dependencies should always be used to get properly updated.

Note

The created map is also used for the pickle file to circumvent the computationally expensive graph and map creation in production mode.

handlers

Module containing the database signal handlers.

The handlers are registered during application startup in apps.ready.

Note

The handlers are not registered in the managment commands makemigrations, migrate and help.

computedfields.handlers.get_old_handler(sender, instance, **kwargs)

get_old_handler handler.

pre_save signal handler to spot incoming fk relation changes. This is needed to correctly update old relations after fk changes, that would contain dirty computed field values after a save. The actual updates on old relations are done during post_save.

computedfields.handlers.postsave_handler(sender, instance, **kwargs)

post_save handler.

Directly updates dependent objects. Does nothing during fixtures.

computedfields.handlers.predelete_handler(sender, instance, **kwargs)

pre_delete handler.

Gets all dependent objects as pk lists and saves them in thread local storage.

computedfields.handlers.postdelete_handler(sender, instance, **kwargs)

post_delete handler.

Loads the dependent objects from the previously saved pk lists and updates them.

computedfields.handlers.m2m_handler(sender, instance, **kwargs)

m2m_change handler.

Works like the other handlers but on the corresponding m2m actions.

Note

The handler triggers updates for both ends of the m2m relation, which might lead to massive updates and thus heavy time consuming database interaction.