seed package

Submodules

seed.decorators module

:copyright (c) 2014 - 2015, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Department of Energy) and contributors. All rights reserved. # NOQA :author

seed.decorators.DecoratorMixin(decorator)

Converts a decorator written for a function view into a mixin for a class-based view.

LoginRequiredMixin = DecoratorMixin(login_required)
class MyView(LoginRequiredMixin):
    pass
class SomeView(DecoratorMixin(some_decorator),
               DecoratorMixin(something_else)):
    pass
seed.decorators.ajax_request(func)
  • Copied from django-annoying, with a small modification. Now we also check
  • for ‘status’ or ‘success’ keys and return correct status codes

If view returned serializable dict, returns response in a format requested by HTTP_ACCEPT header. Defaults to JSON if none requested or match.

Currently supports JSON or YAML (if installed), but can easily be extended.

example:

@ajax_request def my_view(request):

news = News.objects.all() news_titles = [entry.title for entry in news] return {‘news_titles’: news_titles}
seed.decorators.get_prog_key(func_name, import_file_pk)

Return the progress key for the cache

seed.decorators.lock_and_track(fn, *args, **kwargs)

Decorator to lock tasks to single executor and provide progress url.

seed.exporter module

seed.factory module

:copyright (c) 2014 - 2015, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Department of Energy) and contributors. All rights reserved. # NOQA :author

class seed.factory.SEEDFactory

Bases: seed.test_helpers.factory.helpers.DjangoFunctionalFactory

model factory for SEED

classmethod building_snapshot(canonical_building=None, *args, **kwargs)

creates an BuildingSnapshot inst.

if canonical_building (CanonicalBuilding inst.) is None, then a CanonicalBuilding inst. is created and a BuildingSnapshot inst. is created and linked to the CanonicalBuilding inst.

Usage:
ab = SEEDFactory.assessed_building() cb = ab.canonical_building b_snapshot = cb.canonical_snapshot print ab.year_built == b_snapshot.year_built # True
or loop through to create a whole bunch:
for i in range(10):
SEEDFactory.building_snapshot(name=’tester_‘ % i)

seed.models module

:copyright (c) 2014 - 2015, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Department of Energy) and contributors. All rights reserved. # NOQA :author

class seed.models.AttributeOption(*args, **kwargs)

Bases: django.db.models.base.Model

Holds a single conflicting value for a BuildingSnapshot attribute.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception AttributeOption.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

AttributeOption.building_variant
AttributeOption.get_value_source_display(*moreargs, **morekwargs)
AttributeOption.objects = <django.db.models.manager.Manager object at 0x7fb8a0af1e90>
class seed.models.BuildingAttributeVariant(*args, **kwargs)

Bases: django.db.models.base.Model

Place to keep the options of BuildingSnapshot attribute variants.

When we want to select which source’s values should sit in the Canonical Building’s position, we need to draw from a set of options determined during the matching phase. We should only have one ‘Variant’ container per field_name, per snapshot.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception BuildingAttributeVariant.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

BuildingAttributeVariant.building_snapshot
BuildingAttributeVariant.objects = <django.db.models.manager.Manager object at 0x7fb8a0afb390>
BuildingAttributeVariant.options
class seed.models.BuildingSnapshot(*args, **kwargs)

Bases: django_extensions.db.models.TimeStampedModel

The periodical composite of a building from disparate data sources.

Represents the best data between all the datasources for a given building, potentially merged together with other BuildingSnapshot instances’ attribute values.

Two BuildingSnapshots can create a child, forming a match between buildings. Thusly, a BuildingSnapshot’s co-parent is the other parent of its child. The m2m field children with related name parents allow the traversal of the tree. A BuildingSnapshot can have one parent in the case where an edit to data was initiated by a user, and the original field is preserved (treating BuildingSnapshots as immutable objects) and a new BuildingSnapshot is created with the change.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception BuildingSnapshot.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

BuildingSnapshot.address_line_1_source
BuildingSnapshot.address_line_2_source
BuildingSnapshot.best_guess_canonical_building
BuildingSnapshot.block_number_source
BuildingSnapshot.building_certification_source
BuildingSnapshot.building_count_source
BuildingSnapshot.canonical_building
BuildingSnapshot.canonical_for_ds
BuildingSnapshot.canonicalbuilding_set
BuildingSnapshot.child_tree

Recurse to give us a merge-order list of children.

BuildingSnapshot.children
BuildingSnapshot.city_source
BuildingSnapshot.clean(*args, **kwargs)
BuildingSnapshot.co_parent

returns the first co-parent as a BuildingSnapshot inst

BuildingSnapshot.co_parents

returns co-parents for a BuildingSnapshot as a queryset

BuildingSnapshot.conditioned_floor_area_source
BuildingSnapshot.custom_id_1_source
BuildingSnapshot.district_source
BuildingSnapshot.duplicate
BuildingSnapshot.energy_alerts_source
BuildingSnapshot.energy_score_source
BuildingSnapshot.extra_data

A placeholder class that provides a way to set the attribute on the model.

BuildingSnapshot.extra_data_sources

A placeholder class that provides a way to set the attribute on the model.

BuildingSnapshot.generation_date_source
BuildingSnapshot.get_match_type_display(*moreargs, **morekwargs)
BuildingSnapshot.get_next_by_created(*moreargs, **morekwargs)
BuildingSnapshot.get_next_by_modified(*moreargs, **morekwargs)
BuildingSnapshot.get_previous_by_created(*moreargs, **morekwargs)
BuildingSnapshot.get_previous_by_modified(*moreargs, **morekwargs)
BuildingSnapshot.get_source_type_display(*moreargs, **morekwargs)
BuildingSnapshot.gross_floor_area_source
BuildingSnapshot.has_children
BuildingSnapshot.import_file
BuildingSnapshot.last_modified_by
BuildingSnapshot.lot_number_source
BuildingSnapshot.meters
BuildingSnapshot.objects = <seed.managers.json.JsonManager object at 0x7fb8a0ad31d0>
BuildingSnapshot.occupied_floor_area_source
BuildingSnapshot.owner_address_source
BuildingSnapshot.owner_city_state_source
BuildingSnapshot.owner_email_source
BuildingSnapshot.owner_postal_code_source
BuildingSnapshot.owner_source
BuildingSnapshot.owner_telephone_source
BuildingSnapshot.parent_tree

Recurse to give us merge-order list of parents.

BuildingSnapshot.parents
BuildingSnapshot.pm_property_id_source
BuildingSnapshot.postal_code_source
BuildingSnapshot.project_building_snapshots
BuildingSnapshot.project_set
BuildingSnapshot.property_name_source
BuildingSnapshot.property_notes_source
BuildingSnapshot.recent_sale_date_source
BuildingSnapshot.recurse_tree(attr)

Recurse M2M relationship tree, extending list as we go.

Parameters:attr – str, name of attribute we wish to traverse. .e.g. ‘children’, or ‘parents’
BuildingSnapshot.release_date_source
BuildingSnapshot.save(*args, **kwargs)
BuildingSnapshot.site_eui_source
BuildingSnapshot.site_eui_weather_normalized_source
BuildingSnapshot.source_eui_source
BuildingSnapshot.source_eui_weather_normalized_source
BuildingSnapshot.space_alerts_source
BuildingSnapshot.state_province_source
BuildingSnapshot.super_organization
BuildingSnapshot.tax_lot_id_source
BuildingSnapshot.tip

returns the tip (leaf) of the BuildingSnapshot tree

BuildingSnapshot.to_dict(fields=None)

Returns a dict version of this building, either with all fields or masked to just those requested.

BuildingSnapshot.use_description_source
BuildingSnapshot.variants
BuildingSnapshot.year_built_source
BuildingSnapshot.year_ending_source
class seed.models.CanonicalBuilding(*args, **kwargs)

Bases: django.db.models.base.Model

One Table to rule them all, One Table to find them, One Table to bring them all and in the database bind them.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception CanonicalBuilding.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

CanonicalBuilding.audit_logs

This class provides the functionality that makes the related-object managers available as attributes on a model class, for fields that have multiple “remote” values and have a GenericRelation defined in their model (rather than having another model pointed at them). In the example “article.publications”, the publications attribute is a ReverseGenericRelatedObjectsDescriptor instance.

CanonicalBuilding.best_guess
CanonicalBuilding.buildingsnapshot_set
CanonicalBuilding.canonical_snapshot
CanonicalBuilding.labels
CanonicalBuilding.objects = <seed.models.CanonicalManager object at 0x7fb8a0ba6c10>
CanonicalBuilding.raw_objects = <django.db.models.manager.Manager object at 0x7fb8a0ba6c50>
class seed.models.CanonicalManager

Bases: django.db.models.manager.Manager

Manager to add useful model filtering methods

get_query_set(*args, **kwargs)
get_queryset()

Return only active CanonicalBuilding rows.

class seed.models.Column(*args, **kwargs)

Bases: django.db.models.base.Model

The name of a column for a given organization.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception Column.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Column.enum
Column.mapped_mappings
Column.objects = <django.db.models.manager.Manager object at 0x7fb8a0b8d9d0>
Column.organization
Column.raw_mappings
Column.schemas
Column.unit
class seed.models.ColumnMapping(*args, **kwargs)

Bases: django.db.models.base.Model

Stores previous user-defined column mapping.

We’ll pull from this when pulling from varied, dynamic source data to present the user with previous choices for that same field in subsequent data loads.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception ColumnMapping.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

ColumnMapping.column_mapped
ColumnMapping.column_raw
ColumnMapping.get_source_type_display(*moreargs, **morekwargs)
ColumnMapping.is_concatenated()

Returns True if the ColumnMapping represents the concatenation of imported column names; else returns False.

ColumnMapping.is_direct()

Returns True if the ColumnMapping is a direct mapping from imported column name to either a BEDES column or a previously imported column. Returns False if the ColumnMapping represents a concatenation.

ColumnMapping.objects = <django.db.models.manager.Manager object at 0x7fb8a0b9ae10>
ColumnMapping.remove_duplicates(qs, m2m_type='column_raw')

Remove any other Column Mappings that use these columns.

Parameters:
  • qs – queryset of Column. These are the Columns in a M2M with this instance.
  • m2m_type – str, the name of the field we’re comparing against. Defaults to ‘column_raw’.
ColumnMapping.save(*args, **kwargs)

Overrides default model save to eliminate duplicate mappings.

Warning

Other column mappings which have the same raw_columns in them will be removed!

ColumnMapping.super_organization
ColumnMapping.user
class seed.models.Compliance(*args, **kwargs)

Bases: django_extensions.db.models.TimeStampedModel

Compliance(id, created, modified, compliance_type, start_date, end_date, deadline_date, project_id)

AUDITING_COMPLIANCE_CHOICE = 'Auditing'
BENCHMARK_COMPLIANCE_CHOICE = 'Benchmarking'
COMPLIANCE_CHOICES = (('Benchmarking', <django.utils.functional.__proxy__ object at 0x7fb8a0c68f50>), ('Auditing', <django.utils.functional.__proxy__ object at 0x7fb8a0c75190>), ('Retro Commissioning', <django.utils.functional.__proxy__ object at 0x7fb8a0c751d0>))
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception Compliance.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Compliance.RETRO_COMMISSIONING_COMPLIANCE_CHOICE = 'Retro Commissioning'
Compliance.get_compliance_type_display(*moreargs, **morekwargs)
Compliance.get_next_by_created(*moreargs, **morekwargs)
Compliance.get_next_by_modified(*moreargs, **morekwargs)
Compliance.get_previous_by_created(*moreargs, **morekwargs)
Compliance.get_previous_by_modified(*moreargs, **morekwargs)
Compliance.objects = <django.db.models.manager.Manager object at 0x7fb8a0c75b50>
Compliance.project
Compliance.to_dict()
class seed.models.CustomBuildingHeaders(*args, **kwargs)

Bases: django.db.models.base.Model

Specify custom building header mapping for display.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception CustomBuildingHeaders.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

CustomBuildingHeaders.building_headers

A placeholder class that provides a way to set the attribute on the model.

CustomBuildingHeaders.objects = <seed.managers.json.JsonManager object at 0x7fb8a0c75c90>
CustomBuildingHeaders.super_organization
class seed.models.Enum(*args, **kwargs)

Bases: django.db.models.base.Model

Defines a set of enumerated types for a column.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception Enum.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Enum.column_set
Enum.enum_values
Enum.objects = <django.db.models.manager.Manager object at 0x7fb8a0b8d1d0>
class seed.models.EnumValue(*args, **kwargs)

Bases: django.db.models.base.Model

Individual Enumerated Type values.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception EnumValue.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

EnumValue.objects = <django.db.models.manager.Manager object at 0x7fb8a0c7f810>
EnumValue.values
class seed.models.Meter(*args, **kwargs)

Bases: django.db.models.base.Model

Meter specific attributes.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception Meter.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Meter.building_snapshot
Meter.get_energy_type_display(*moreargs, **morekwargs)
Meter.get_energy_units_display(*moreargs, **morekwargs)
Meter.objects = <django.db.models.manager.Manager object at 0x7fb8a0afbf10>
Meter.timeseries_data
class seed.models.NonCanonicalProjectBuildings(*args, **kwargs)

Bases: django.db.models.base.Model

Holds a reference to all project buildings that do not point at a canonical building snapshot.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception NonCanonicalProjectBuildings.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

NonCanonicalProjectBuildings.objects = <django.db.models.manager.Manager object at 0x7fb8a0af1a50>
NonCanonicalProjectBuildings.projectbuilding
class seed.models.Project(*args, **kwargs)

Bases: django_extensions.db.models.TimeStampedModel

Project(id, created, modified, name, slug, owner_id, last_modified_by_id, super_organization_id, description, status)

ACTIVE_STATUS = 1
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

Project.INACTIVE_STATUS = 0
exception Project.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Project.STATUS_CHOICES = ((0, <django.utils.functional.__proxy__ object at 0x7fb8a0c56d10>), (1, <django.utils.functional.__proxy__ object at 0x7fb8a0c56d90>))
Project.adding_buildings_status_percentage_cache_key
Project.building_snapshots
Project.compliance_set
Project.get_compliance()
Project.get_next_by_created(*moreargs, **morekwargs)
Project.get_next_by_modified(*moreargs, **morekwargs)
Project.get_previous_by_created(*moreargs, **morekwargs)
Project.get_previous_by_modified(*moreargs, **morekwargs)
Project.get_status_display(*moreargs, **morekwargs)
Project.has_compliance
Project.last_modified_by
Project.objects = <django.db.models.manager.Manager object at 0x7fb8a0c6ec50>
Project.owner
Project.project_building_snapshots
Project.removing_buildings_status_percentage_cache_key
Project.super_organization
Project.to_dict()
class seed.models.ProjectBuilding(*args, **kwargs)

Bases: django_extensions.db.models.TimeStampedModel

ProjectBuilding(id, created, modified, building_snapshot_id, project_id, compliant, approved_date, approver_id)

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception ProjectBuilding.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

ProjectBuilding.approver
ProjectBuilding.building_snapshot
ProjectBuilding.get_next_by_created(*moreargs, **morekwargs)
ProjectBuilding.get_next_by_modified(*moreargs, **morekwargs)
ProjectBuilding.get_previous_by_created(*moreargs, **morekwargs)
ProjectBuilding.get_previous_by_modified(*moreargs, **morekwargs)
ProjectBuilding.noncanonicalprojectbuildings_set
ProjectBuilding.objects = <django.db.models.manager.Manager object at 0x7fb8a0c68610>
ProjectBuilding.project
ProjectBuilding.to_dict()
class seed.models.Schema(*args, **kwargs)

Bases: django.db.models.base.Model

Groups ColumnMappings together for identification later.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception Schema.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Schema.columns
Schema.objects = <django.db.models.manager.Manager object at 0x7fb8a0ba6a10>
Schema.organization
class seed.models.StatusLabel(*args, **kwargs)

Bases: django_extensions.db.models.TimeStampedModel

StatusLabel(id, created, modified, name, color, super_organization_id)

BLUE_CHOICE = 'blue'
COLOR_CHOICES = (('red', <django.utils.functional.__proxy__ object at 0x7fb8a0c68490>), ('blue', <django.utils.functional.__proxy__ object at 0x7fb8a0c68690>), ('light blue', <django.utils.functional.__proxy__ object at 0x7fb8a0c686d0>), ('green', <django.utils.functional.__proxy__ object at 0x7fb8a0c68710>), ('white', <django.utils.functional.__proxy__ object at 0x7fb8a0c68750>), ('orange', <django.utils.functional.__proxy__ object at 0x7fb8a0c68790>), ('gray', <django.utils.functional.__proxy__ object at 0x7fb8a0c687d0>))
DEFAULT_LABELS = ['Residential', 'Non-Residential', 'Violation', 'Compliant', 'Missing Data', 'Questionable Report', 'Update Bldg Info', 'Call', 'Email', 'High EUI', 'Low EUI', 'Exempted', 'Extension', 'Change of Ownership']
exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

StatusLabel.GRAY_CHOICE = 'gray'
StatusLabel.GREEN_CHOICE = 'green'
StatusLabel.LIGHT_BLUE_CHOICE = 'light blue'
exception StatusLabel.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

StatusLabel.ORANGE_CHOICE = 'orange'
StatusLabel.RED_CHOICE = 'red'
StatusLabel.WHITE_CHOICE = 'white'
StatusLabel.canonicalbuilding_set
StatusLabel.get_color_display(*moreargs, **morekwargs)
StatusLabel.get_next_by_created(*moreargs, **morekwargs)
StatusLabel.get_next_by_modified(*moreargs, **morekwargs)
StatusLabel.get_previous_by_created(*moreargs, **morekwargs)
StatusLabel.get_previous_by_modified(*moreargs, **morekwargs)
StatusLabel.objects = <django.db.models.manager.Manager object at 0x7fb8a0c75110>
StatusLabel.super_organization
StatusLabel.to_dict()
class seed.models.TimeSeries(*args, **kwargs)

Bases: django.db.models.base.Model

For storing engergy use over time.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception TimeSeries.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

TimeSeries.meter
TimeSeries.objects = <django.db.models.manager.Manager object at 0x7fb8a0b06490>
class seed.models.Unit(*args, **kwargs)

Bases: django.db.models.base.Model

Unit of measure for a Column Value.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception Unit.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Unit.column_set
Unit.get_unit_type_display(*moreargs, **morekwargs)
Unit.objects = <django.db.models.manager.Manager object at 0x7fb8a0c7f4d0>
seed.models.clean_canonicals(b1, b2, new_snapshot)

Make sure that we don’t leave dead limbs in our tree.

Parameters:
  • b1 – BuildingSnapshot, parent 1
  • b2 – BuildingSnapshot, parent 2
  • new_snapshot – BuildingSnapshot, child.
seed.models.find_canonical_building_values(org)

Get all canonical building snapshots’ id info for an organization.

Parameters:org – Organization inst.
Return type:list of tuples, field values specified in BS_VALUES_LIST for all canonical buildings related to an organization.

NB: This does not return a queryset!

seed.models.find_unmatched_buildings(import_file)

Get unmatched building snapshots’ id info from an import file.

Parameters:import_file – ImportFile inst.
Return type:list of tuples, field values specified in BS_VALUES_LIST.

NB: This does not return a queryset!

seed.models.get_ancestors(building)

gets all the non-raw, non-composite ancestors of a building

Recursive function to traverse the tree upward. source_type {

2: ASSESSED_BS, 3: PORTFOLIO_BS, 4: COMPOSITE_BS, 6: GREEN_BUTTON_BS

}

Parameters:building – BuildingSnapshot inst.
Returns:list of BuildingSnapshot inst., ancestors of building
seed.models.get_column_mapping(column_raw, organization, attr_name='column_mapped')

Callable provided to MCM to return a previously mapped field.

Parameters:
  • column_raw – str, the column name of the raw data.
  • organization – Organization inst.
  • attr_name – str, name of attribute on ColumnMapping to pull out. whether we’re looking at a mapping from the perspective of a raw_column (like we do when creating a mapping), or mapped_column, (like when we’re applying that mapping).
Returns:

list of mapped items, float representation of confidence.
seed.models.get_column_mappings(organization)

Returns dict of all the column mappings for an Org’s given source type

Parameters:organization – inst, Organization.
Returns dict, list of dict:
 

Use this when actually performing mapping between datasources, but only call it after all of the mappings have been saved to the ColumnMapping table.

seed.models.get_or_create_canonical(b1, b2=None)

Gets most trusted Canonical Building.

Parameters:
  • b1 – BuildingSnapshot model type.
  • b2 – BuildingSnapshot model type.
Return type:

CanonicalBuilding inst. Will contain PK.

NB: preference is given to existing snapshots’ Canonical link.

seed.models.get_sourced_attributes(snapshot)

Return all the attribute names that get sourced.

seed.models.initialize_canonical_building(snapshot, user_pk)

Called to create a Canonicalbuilding from a single snapshot.

Parameters:
  • snapshot – BuildingSnapshot inst.
  • user_pk – The user id of the user initiating the CanonicalBuilding
seed.models.obj_to_dict(obj)

serializes obj for a JSON friendly version tries to serialize JsonField

seed.models.save_column_names(bs, mapping=None)

Save unique column names for extra_data in this organization.

Basically this is a record of all the extra_data keys we’ve ever seen for a particular organization.

Parameters:bs – BuildingSnapshot instance.
seed.models.save_snapshot_match(b1_pk, b2_pk, confidence=None, user=None, match_type=None, default_pk=None)

Saves a match between two models as a new snapshot; updates Canonical.

Parameters:
  • b1_pk – int, id for building snapshot.
  • b2_pk – int, id for building snapshot.
  • confidence – (optional) float, likelihood that two models are linked.
  • user – (optional) User inst, last_modified_by for BuildingSnapshot.
Return type:

BuildingSnapshot instance, post save.

Determines which Canonical link should be used. If canonical is specified, we’re probably changing a building’s Canonical link, so use that Canonical Building. Otherwise, use the model we match against. If none exists, create it.

Update mapped fields in the new snapshot, update canonical links.

seed.models.set_initial_sources(snapshot)

Sets the PK for the original sources to self.

seed.models.unmatch_snapshot_tree(building_pk)

May or may not obviate unmatch_snapshot. Experimental.

Parameters:building_pk – int - Primary Key for a BuildingSnapshot.

Warning

unmatch_snapshot_tree potentially modifies years of merged data. Anything descended from the building_pk will be deleted. The intent is to completely separate building_pk‘s influence on the resultant canonical_snapshot. The user is saying that these are separate entities afterall, yes?

Basically, this function works by getting a merge order list of children from the perspective of building_pk and a list of parents from the perspective of leaf node in the child tree. We take the difference between these lists and call that the remaining_ancestors from which we reconstruct the merge tree for our CanonicalBuilding.

building_pk either gets a reactivated CanonicalBuilding, or a new one.

seed.models.update_building(old_snapshot, updated_values, user, *args, **kwargs)

Creates a new snapshot with updated values.

seed.reconcile module

:copyright (c) 2014 - 2015, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Department of Energy) and contributors. All rights reserved. # NOQA :author

seed.reconcile.build_q(model_a, a_attr, model_b_class, b_attr, a_value=None)

build Q objects for model_a in model_b_class among attrs.

Parameters:
  • model_a – model instance.
  • a_attr – str, attribute name on model_a whose value we search for.
  • model_b_class – class, the object class we filter through.
  • b_attr – str, the attribute name we’re searching.
seed.reconcile.build_q_filter(q_set, model_a, a_attr, model_b_class, b_attr, op, trans)

Build a set of Q objects to filter table results.

Parameters:
  • q_set – the Q object set that’s being built cumulatively.
  • model_a – model instance.
  • a_attr – str, the name of an attribute we’re querying from.
  • model_b_class – class, model class we’re querying to.
  • b_attr – str, model attribute we’re querying to (on model_b_class).
  • op – callable, takes two parameters. This callable should be an operator module function, e.g. operator.ior, or operator.iand.
  • trans – callable or None. If callable, we apply this callable to

our a_attr for circumstances in which we need to break up its value into sub values for more accurate querying (e.g. address data.).

seed.reconcile.calculate_confidence(model_a, model_b, mapping='FIRST_PASS')

Determine the similarity between model_a and model_b.

Goes through the mappings and compares those attrs between each of the modules produced in the search function.

Rtype float:0.0 to 1.0, the degree that the two models are similar.
seed.reconcile.get_best_match(model_a, search_results)

Return highest confidence model_b match and its confidence number.

Parameters:
  • model_a – model instance, the known model.
  • search_results – a queryset of model_b insts returned from search
Rtype tuple:

(float, model_b_inst|None)
seed.reconcile.logger = <celery.utils.log.ProcessAwareLogger object at 0x7fb89c81a190>

Module seeks to provide a general way to tie together information about a single entity from two datasources using sparse data.

Sample usage:

A task or a view that wants to find a correlated building from another data set would execute the following set of calls:

model_a is an instance of a model that I have (probably just recently
loaded).

model_b_qs is the dataset type we’re trying to find matches in.

` possible_matches = search(model_a, model_b_qs) best_match = get_best_match(model_a, possible_matches) best_match >>> (0.89, AssessedBuildingInst) `

seed.reconcile.search(model_a, model_b_qs, mapping='FIRST_PASS', op=<built-in function ior>)

Create and execute filters according to mapping rules.

Parameters:
  • model_a – model instance we’re querying from.
  • model_b_qs – QuerySet, the pre-filtered set of models to match.
  • mapping – str, prefix for a mapping between model_a and model_b.
  • op – callable, how we combine this set of Q objects.
Rtype Queryset:

the Q objects are strung together and passed filter. The results, if any, are the set of possible matches.

seed.search module

:copyright (c) 2014 - 2015, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Department of Energy) and contributors. All rights reserved. # NOQA :author

seed.search.build_json_params(order_by, sort_reverse)

returns db_columns, extra_data_sort, and updated order_by

Parameters:order_by (str) – field to order_by
Returns:tuple: db_columns: dict of known DB columns i.e. non-JSONfFeld, extra_data_sort bool if order_by is in extra_data JsonField, order_by str if sort_reverse and DB column prepend a ‘-‘ for the django order_by
seed.search.build_shared_buildings_orgs(orgs)

returns a list of sibling and parent orgs

seed.search.convert_to_js_timestamp(timestamp)

converts a django/python datetime object to milliseconds since epoch duplicated code with seed utils due to circular imports

seed.search.create_building_queryset(orgs, exclude, order_by, other_orgs=None, extra_data_sort=False)

creates a querset of buildings within orgs. If other_orgs, buildings in both orgs and other_orgs will be represented in the queryset.

Parameters:
  • orgs – queryset of Organization inst.
  • exclude – django query exclude dict.
  • order_by – django query order_by str.
  • other_orgs – list of other orgs to or the query
seed.search.filter_other_params(queryset, other_params, db_columns)

applies a dictionary filter to the query set. Does some domain specific parsing,

mostly to remove extra query params and deal with ranges. Ranges should be passed in as ‘<field name>__lte’ or ‘<field name>__gte’

e.g. other_params = {‘gross_floor_area__lte’: 50000}
Parameters:
  • Queryset queryset (Django) – queryset to be filtered
  • other_params (dict) – dictionary to be parsed and applied to filter.
  • db_columns (dict) – list of column names, extra_data blob outsite these
Returns:

Django Queryset:
seed.search.generate_paginated_results(queryset, number_per_page=25, page=1, whitelist_orgs=None, below_threshold=False)
returns a page of results as a list from the queryset for the given
fields
Usage:
generate_paginated_results(q, 1)

Out: [{

‘gross_floor_area’: 1710, ‘site_eui’: 123, ‘tax_lot_id’: u’IMP9-test001’, ‘year_built’: 2001

}]

Parameters:
  • queryset – optional queryset to filter from
  • number_per_page (int) – optional number of results per page
  • page (int) – optional page of results to get
  • whitelist_orgs – a queryset returning the organizations in which all building fields can be returned, otherwise only the parent organization’s exportable_fields should be returned. The whitelist_orgs are the orgs the request user belongs.
  • below_threshold – True if less than the parent org’s query threshold is greater than the number of queryset results. If True, only return buildings within whitelist_orgs.
seed.search.get_building_fieldnames()

returns a list of field names for the BuildingSnapshot class/model that will be searched against

seed.search.get_orgs_w_public_fields()

returns a list of orgs that have publicly shared fields

seed.search.is_not_whitelist_building(parent_org, building, whitelist_orgs)

returns false if a building is part of the whitelist_orgs

Parameters:
  • parent_org – the umbrella parent Organization inst.
  • building – the BuildingSnapshot inst.
  • whitelist_orgs – queryset of Organization insts.
Returns:

bool
seed.search.mask_results(search_results)

masks (deletes dict keys) for non-shared public fields

seed.search.orchestrate_search_filter_sort(params, user)

Given a parsed set of params, perform the search, filter, and sort for BuildingSnapshot’s

seed.search.paginate_results(request, search_results)

returns a paginated list of dict results

seed.search.parse_body(request)

parses the request body for search params, q, etc

Params request:django wsgi request object
Returns:dict { ‘exclude’: dict, exclude dict for django queryset ‘order_by’: str, query order_by, defaults to ‘tax_lot_id’ ‘sort_reverse’: bool, True if ASC, False if DSC ‘page’: int, pagination page ‘number_per_page’: int, number per pagination page ‘show_shared_buildings’: bool, whether to search across all user’s orgs ‘q’: str, global search param ‘other_search_params’: dict, filter params ‘project_id’: str, project id if exists in body

}

seed.search.process_search_params(params, user, is_api_request=False)

Given a python representation of a search query, process it into the internal format that is used for searching, filtering, sorting, and pagination.

Parameters:
  • params – a python object representing the search query
  • user – the user this search is for
  • is_api_request – bool, boolean whether this search is being done as an api request.
Returns:

dict { ‘exclude’: dict, exclude dict for django queryset ‘order_by’: str, query order_by, defaults to ‘tax_lot_id’ ‘sort_reverse’: bool, True if ASC, False if DSC ‘page’: int, pagination page ‘number_per_page’: int, number per pagination page ‘show_shared_buildings’: bool, whether to search across all user’s orgs ‘q’: str, global search param ‘other_search_params’: dict, filter params ‘project_id’: str, project id if exists in body

}

seed.search.remove_results_below_q_threshold(search_results)

removes buildings if total count of buildings grouped by org is less than their org’s public query threshold

Parameters:search_results (list/queryset) – search results
Returns:list or queryset
seed.search.search_buildings(q, fieldnames=None, queryset=None)

returns a queryset for matching buildings

Parameters:
  • or unicode q (str) – search string
  • fieldnames (list) – list of BuildingSnapshot model fieldnames (defaults to those generated by get_building_field_names())
  • queryset – optional queryset to filter from, defaults to BuildingSnapshot.objects.none()
Returns:

queryset:queryset of matching buildings

seed.search.search_public_buildings(request, orgs)
returns a queryset or list of buildings matching the search params and
count
Parameters:request – wsgi request (Django) for parsing params
Orgs:list of Organization instances to search within
Returns:tuple (search_results_list, result count)

seed.tasks module

:copyright (c) 2014 - 2015, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Department of Energy) and contributors. All rights reserved. # NOQA :author

exception seed.tasks.DuplicateDataError(id)

Bases: exceptions.RuntimeError

seed.tasks.add_cache_increment_parameter(tasks)

This adds the cache increment value to the signature to each subtask.

seed.tasks.apply_data_func(mappable_columns)

Returns a function that captures mappable_types in a closure and will add a key to extra data if not in mappable_types else

seed.tasks.apply_extra_data(model, key, value)

Function sent to MCM to apply mapped columns into extra_data.

seed.tasks.cache_first_rows(import_file, parser)

Cache headers, and rows 2-6 for validation/viewing.

Parameters:
  • import_file – ImportFile inst.
  • parser – unicode-csv.Reader instance.

Unfortunately, this is duplicated logic from data_importer, but since data_importer makes many faulty assumptions we need to do it differently.

seed.tasks.get_canonical_id_matches(org_id, pm_id, tax_id, custom_id)

Returns canonical snapshots that match at least one id.

seed.tasks.get_canonical_snapshots(org_id)

Return all of the BuildingSnapshots that are canonical for an org.

seed.tasks.handle_id_matches(unmatched_bs, import_file, user_pk)

“Deals with exact matches in the IDs of buildings.

seed.tasks.handle_results(results, b_idx, can_rev_idx, unmatched_list, user_pk)

Seek IDs and save our snapshot match.

Parameters:
  • results – list of tuples. [(‘match’, 0.99999),...]
  • b_idx – int, the index of the current building in the unmatched_list.
  • can_rev_idx – dict, reverse index from match -> canonical PK.
  • user_pk – user ID, used for AuditLog logging
Unmatched_list:

list of dicts, the result of a values_list query for unmatched BSes.
seed.tasks.is_same_snapshot(s1, s2)

seed.token_generators module

:copyright (c) 2014 - 2015, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Department of Energy) and contributors. All rights reserved. # NOQA :author

class seed.token_generators.SignupTokenGenerator

Bases: object

Strategy object used to generate and check tokens for the password reset mechanism.

check_token(user, token, token_expires=True)

Check that a password reset token is correct for a given user.

make_token(user)

Returns a token that can be used once to do a password reset for the given user.

seed.urls module

:copyright (c) 2014 - 2015, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Department of Energy) and contributors. All rights reserved. # NOQA :author

seed.utils module

:copyright (c) 2014 - 2015, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Department of Energy) and contributors. All rights reserved. # NOQA :author

seed.views module

:copyright (c) 2014 - 2015, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Department of Energy) and contributors. All rights reserved. # NOQA :author

Module contents

:copyright (c) 2014 - 2015, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Department of Energy) and contributors. All rights reserved. # NOQA :author