seed package¶
Subpackages¶
Submodules¶
seed.decorators module¶
:copyright (c) 2014 - 2015, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Department of Energy) and contributors. All rights reserved. # NOQA :author
- seed.decorators.DecoratorMixin(decorator)¶
Converts a decorator written for a function view into a mixin for a class-based view.
LoginRequiredMixin = DecoratorMixin(login_required) class MyView(LoginRequiredMixin): pass class SomeView(DecoratorMixin(some_decorator), DecoratorMixin(something_else)): pass
- seed.decorators.ajax_request(func)¶
- Copied from django-annoying, with a small modification. Now we also check
- for ‘status’ or ‘success’ keys and return correct status codes
If view returned serializable dict, returns response in a format requested by HTTP_ACCEPT header. Defaults to JSON if none requested or match.
Currently supports JSON or YAML (if installed), but can easily be extended.
example:
@ajax_request def my_view(request):
news = News.objects.all() news_titles = [entry.title for entry in news] return {‘news_titles’: news_titles}
- seed.decorators.get_prog_key(func_name, import_file_pk)¶
Return the progress key for the cache
- seed.decorators.lock_and_track(fn, *args, **kwargs)¶
Decorator to lock tasks to single executor and provide progress url.
seed.exporter module¶
seed.factory module¶
:copyright (c) 2014 - 2015, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Department of Energy) and contributors. All rights reserved. # NOQA :author
- class seed.factory.SEEDFactory¶
Bases: seed.test_helpers.factory.helpers.DjangoFunctionalFactory
model factory for SEED
- classmethod building_snapshot(canonical_building=None, *args, **kwargs)¶
creates an BuildingSnapshot inst.
if canonical_building (CanonicalBuilding inst.) is None, then a CanonicalBuilding inst. is created and a BuildingSnapshot inst. is created and linked to the CanonicalBuilding inst.
- Usage:
- ab = SEEDFactory.assessed_building() cb = ab.canonical_building b_snapshot = cb.canonical_snapshot print ab.year_built == b_snapshot.year_built # True
- or loop through to create a whole bunch:
- for i in range(10):
- SEEDFactory.building_snapshot(name=’tester_‘ % i)
seed.models module¶
:copyright (c) 2014 - 2015, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Department of Energy) and contributors. All rights reserved. # NOQA :author
- class seed.models.AttributeOption(*args, **kwargs)¶
Bases: django.db.models.base.Model
Holds a single conflicting value for a BuildingSnapshot attribute.
- exception DoesNotExist¶
Bases: django.core.exceptions.ObjectDoesNotExist
- exception AttributeOption.MultipleObjectsReturned¶
Bases: django.core.exceptions.MultipleObjectsReturned
- AttributeOption.building_variant¶
- AttributeOption.get_value_source_display(*moreargs, **morekwargs)¶
- AttributeOption.objects = <django.db.models.manager.Manager object at 0x7fc70acdde90>¶
- class seed.models.BuildingAttributeVariant(*args, **kwargs)¶
Bases: django.db.models.base.Model
Place to keep the options of BuildingSnapshot attribute variants.
When we want to select which source’s values should sit in the Canonical Building’s position, we need to draw from a set of options determined during the matching phase. We should only have one ‘Variant’ container per field_name, per snapshot.
- exception DoesNotExist¶
Bases: django.core.exceptions.ObjectDoesNotExist
- exception BuildingAttributeVariant.MultipleObjectsReturned¶
Bases: django.core.exceptions.MultipleObjectsReturned
- BuildingAttributeVariant.building_snapshot¶
- BuildingAttributeVariant.objects = <django.db.models.manager.Manager object at 0x7fc70ace5390>¶
- BuildingAttributeVariant.options¶
- class seed.models.BuildingSnapshot(*args, **kwargs)¶
Bases: django_extensions.db.models.TimeStampedModel
The periodical composite of a building from disparate data sources.
Represents the best data between all the datasources for a given building, potentially merged together with other BuildingSnapshot instances’ attribute values.
Two BuildingSnapshots can create a child, forming a match between buildings. Thusly, a BuildingSnapshot’s co-parent is the other parent of its child. The m2m field children with related name parents allow the traversal of the tree. A BuildingSnapshot can have one parent in the case where an edit to data was initiated by a user, and the original field is preserved (treating BuildingSnapshots as immutable objects) and a new BuildingSnapshot is created with the change.
- exception DoesNotExist¶
Bases: django.core.exceptions.ObjectDoesNotExist
- exception BuildingSnapshot.MultipleObjectsReturned¶
Bases: django.core.exceptions.MultipleObjectsReturned
- BuildingSnapshot.address_line_1_source¶
- BuildingSnapshot.address_line_2_source¶
- BuildingSnapshot.best_guess_canonical_building¶
- BuildingSnapshot.block_number_source¶
- BuildingSnapshot.building_certification_source¶
- BuildingSnapshot.building_count_source¶
- BuildingSnapshot.canonical_building¶
- BuildingSnapshot.canonical_for_ds¶
- BuildingSnapshot.canonicalbuilding_set¶
- BuildingSnapshot.child_tree¶
Recurse to give us a merge-order list of children.
- BuildingSnapshot.children¶
- BuildingSnapshot.city_source¶
- BuildingSnapshot.clean(*args, **kwargs)¶
- BuildingSnapshot.co_parent¶
returns the first co-parent as a BuildingSnapshot inst
- BuildingSnapshot.co_parents¶
returns co-parents for a BuildingSnapshot as a queryset
- BuildingSnapshot.conditioned_floor_area_source¶
- BuildingSnapshot.custom_id_1_source¶
- BuildingSnapshot.district_source¶
- BuildingSnapshot.duplicate¶
- BuildingSnapshot.energy_alerts_source¶
- BuildingSnapshot.energy_score_source¶
- BuildingSnapshot.extra_data¶
A placeholder class that provides a way to set the attribute on the model.
- BuildingSnapshot.extra_data_sources¶
A placeholder class that provides a way to set the attribute on the model.
- BuildingSnapshot.generation_date_source¶
- BuildingSnapshot.get_match_type_display(*moreargs, **morekwargs)¶
- BuildingSnapshot.get_next_by_created(*moreargs, **morekwargs)¶
- BuildingSnapshot.get_next_by_modified(*moreargs, **morekwargs)¶
- BuildingSnapshot.get_previous_by_created(*moreargs, **morekwargs)¶
- BuildingSnapshot.get_previous_by_modified(*moreargs, **morekwargs)¶
- BuildingSnapshot.get_source_type_display(*moreargs, **morekwargs)¶
- BuildingSnapshot.gross_floor_area_source¶
- BuildingSnapshot.has_children¶
- BuildingSnapshot.import_file¶
- BuildingSnapshot.last_modified_by¶
- BuildingSnapshot.lot_number_source¶
- BuildingSnapshot.meters¶
- BuildingSnapshot.objects = <seed.managers.json.JsonManager object at 0x7fc70acbf1d0>¶
- BuildingSnapshot.occupied_floor_area_source¶
- BuildingSnapshot.owner_address_source¶
- BuildingSnapshot.owner_city_state_source¶
- BuildingSnapshot.owner_email_source¶
- BuildingSnapshot.owner_postal_code_source¶
- BuildingSnapshot.owner_source¶
- BuildingSnapshot.owner_telephone_source¶
- BuildingSnapshot.parent_tree¶
Recurse to give us merge-order list of parents.
- BuildingSnapshot.parents¶
- BuildingSnapshot.pm_property_id_source¶
- BuildingSnapshot.postal_code_source¶
- BuildingSnapshot.project_building_snapshots¶
- BuildingSnapshot.project_set¶
- BuildingSnapshot.property_name_source¶
- BuildingSnapshot.property_notes_source¶
- BuildingSnapshot.recent_sale_date_source¶
- BuildingSnapshot.recurse_tree(attr)¶
Recurse M2M relationship tree, extending list as we go.
Parameters: attr – str, name of attribute we wish to traverse. .e.g. ‘children’, or ‘parents’
- BuildingSnapshot.release_date_source¶
- BuildingSnapshot.save(*args, **kwargs)¶
- BuildingSnapshot.site_eui_source¶
- BuildingSnapshot.site_eui_weather_normalized_source¶
- BuildingSnapshot.source_eui_source¶
- BuildingSnapshot.source_eui_weather_normalized_source¶
- BuildingSnapshot.space_alerts_source¶
- BuildingSnapshot.state_province_source¶
- BuildingSnapshot.super_organization¶
- BuildingSnapshot.tax_lot_id_source¶
- BuildingSnapshot.tip¶
returns the tip (leaf) of the BuildingSnapshot tree
- BuildingSnapshot.to_dict(fields=None)¶
Returns a dict version of this building, either with all fields or masked to just those requested.
- BuildingSnapshot.use_description_source¶
- BuildingSnapshot.variants¶
- BuildingSnapshot.year_built_source¶
- BuildingSnapshot.year_ending_source¶
- class seed.models.CanonicalBuilding(*args, **kwargs)¶
Bases: django.db.models.base.Model
One Table to rule them all, One Table to find them, One Table to bring them all and in the database bind them.
- exception DoesNotExist¶
Bases: django.core.exceptions.ObjectDoesNotExist
- exception CanonicalBuilding.MultipleObjectsReturned¶
Bases: django.core.exceptions.MultipleObjectsReturned
- CanonicalBuilding.audit_logs¶
This class provides the functionality that makes the related-object managers available as attributes on a model class, for fields that have multiple “remote” values and have a GenericRelation defined in their model (rather than having another model pointed at them). In the example “article.publications”, the publications attribute is a ReverseGenericRelatedObjectsDescriptor instance.
- CanonicalBuilding.best_guess¶
- CanonicalBuilding.buildingsnapshot_set¶
- CanonicalBuilding.canonical_snapshot¶
- CanonicalBuilding.labels¶
- CanonicalBuilding.objects = <seed.models.CanonicalManager object at 0x7fc70ad92c10>¶
- CanonicalBuilding.raw_objects = <django.db.models.manager.Manager object at 0x7fc70ad92c50>¶
- class seed.models.CanonicalManager¶
Bases: django.db.models.manager.Manager
Manager to add useful model filtering methods
- get_query_set(*args, **kwargs)¶
- get_queryset()¶
Return only active CanonicalBuilding rows.
- class seed.models.Column(*args, **kwargs)¶
Bases: django.db.models.base.Model
The name of a column for a given organization.
- exception DoesNotExist¶
Bases: django.core.exceptions.ObjectDoesNotExist
- exception Column.MultipleObjectsReturned¶
Bases: django.core.exceptions.MultipleObjectsReturned
- Column.enum¶
- Column.mapped_mappings¶
- Column.objects = <django.db.models.manager.Manager object at 0x7fc70ad7b9d0>¶
- Column.organization¶
- Column.raw_mappings¶
- Column.schemas¶
- Column.unit¶
- class seed.models.ColumnMapping(*args, **kwargs)¶
Bases: django.db.models.base.Model
Stores previous user-defined column mapping.
We’ll pull from this when pulling from varied, dynamic source data to present the user with previous choices for that same field in subsequent data loads.
- exception DoesNotExist¶
Bases: django.core.exceptions.ObjectDoesNotExist
- exception ColumnMapping.MultipleObjectsReturned¶
Bases: django.core.exceptions.MultipleObjectsReturned
- ColumnMapping.column_mapped¶
- ColumnMapping.column_raw¶
- ColumnMapping.get_source_type_display(*moreargs, **morekwargs)¶
- ColumnMapping.is_concatenated()¶
Returns True if the ColumnMapping represents the concatenation of imported column names; else returns False.
- ColumnMapping.is_direct()¶
Returns True if the ColumnMapping is a direct mapping from imported column name to either a BEDES column or a previously imported column. Returns False if the ColumnMapping represents a concatenation.
- ColumnMapping.objects = <django.db.models.manager.Manager object at 0x7fc70ad85e10>¶
- ColumnMapping.remove_duplicates(qs, m2m_type='column_raw')¶
Remove any other Column Mappings that use these columns.
Parameters: - qs – queryset of Column. These are the Columns in a M2M with this instance.
- m2m_type – str, the name of the field we’re comparing against. Defaults to ‘column_raw’.
- ColumnMapping.save(*args, **kwargs)¶
Overrides default model save to eliminate duplicate mappings.
Warning
Other column mappings which have the same raw_columns in them will be removed!
- ColumnMapping.super_organization¶
- ColumnMapping.user¶
- class seed.models.Compliance(*args, **kwargs)¶
Bases: django_extensions.db.models.TimeStampedModel
Compliance(id, created, modified, compliance_type, start_date, end_date, deadline_date, project_id)
- AUDITING_COMPLIANCE_CHOICE = 'Auditing'¶
- BENCHMARK_COMPLIANCE_CHOICE = 'Benchmarking'¶
- COMPLIANCE_CHOICES = (('Benchmarking', <django.utils.functional.__proxy__ object at 0x7fc70ae56f50>), ('Auditing', <django.utils.functional.__proxy__ object at 0x7fc70ae63190>), ('Retro Commissioning', <django.utils.functional.__proxy__ object at 0x7fc70ae631d0>))¶
- exception DoesNotExist¶
Bases: django.core.exceptions.ObjectDoesNotExist
- exception Compliance.MultipleObjectsReturned¶
Bases: django.core.exceptions.MultipleObjectsReturned
- Compliance.RETRO_COMMISSIONING_COMPLIANCE_CHOICE = 'Retro Commissioning'¶
- Compliance.get_compliance_type_display(*moreargs, **morekwargs)¶
- Compliance.get_next_by_created(*moreargs, **morekwargs)¶
- Compliance.get_next_by_modified(*moreargs, **morekwargs)¶
- Compliance.get_previous_by_created(*moreargs, **morekwargs)¶
- Compliance.get_previous_by_modified(*moreargs, **morekwargs)¶
- Compliance.objects = <django.db.models.manager.Manager object at 0x7fc70ae63b50>¶
- Compliance.project¶
- Compliance.to_dict()¶
- class seed.models.CustomBuildingHeaders(*args, **kwargs)¶
Bases: django.db.models.base.Model
Specify custom building header mapping for display.
- exception DoesNotExist¶
Bases: django.core.exceptions.ObjectDoesNotExist
- exception CustomBuildingHeaders.MultipleObjectsReturned¶
Bases: django.core.exceptions.MultipleObjectsReturned
- CustomBuildingHeaders.building_headers¶
A placeholder class that provides a way to set the attribute on the model.
- CustomBuildingHeaders.objects = <seed.managers.json.JsonManager object at 0x7fc70ae63c90>¶
- CustomBuildingHeaders.super_organization¶
- class seed.models.Enum(*args, **kwargs)¶
Bases: django.db.models.base.Model
Defines a set of enumerated types for a column.
- exception DoesNotExist¶
Bases: django.core.exceptions.ObjectDoesNotExist
- exception Enum.MultipleObjectsReturned¶
Bases: django.core.exceptions.MultipleObjectsReturned
- Enum.column_set¶
- Enum.enum_values¶
- Enum.objects = <django.db.models.manager.Manager object at 0x7fc70ad7b1d0>¶
- class seed.models.EnumValue(*args, **kwargs)¶
Bases: django.db.models.base.Model
Individual Enumerated Type values.
- exception DoesNotExist¶
Bases: django.core.exceptions.ObjectDoesNotExist
- exception EnumValue.MultipleObjectsReturned¶
Bases: django.core.exceptions.MultipleObjectsReturned
- EnumValue.objects = <django.db.models.manager.Manager object at 0x7fc70ae6b810>¶
- EnumValue.values¶
- class seed.models.Meter(*args, **kwargs)¶
Bases: django.db.models.base.Model
Meter specific attributes.
- exception DoesNotExist¶
Bases: django.core.exceptions.ObjectDoesNotExist
- exception Meter.MultipleObjectsReturned¶
Bases: django.core.exceptions.MultipleObjectsReturned
- Meter.building_snapshot¶
- Meter.get_energy_type_display(*moreargs, **morekwargs)¶
- Meter.get_energy_units_display(*moreargs, **morekwargs)¶
- Meter.objects = <django.db.models.manager.Manager object at 0x7fc70ace5f10>¶
- Meter.timeseries_data¶
- class seed.models.NonCanonicalProjectBuildings(*args, **kwargs)¶
Bases: django.db.models.base.Model
Holds a reference to all project buildings that do not point at a canonical building snapshot.
- exception DoesNotExist¶
Bases: django.core.exceptions.ObjectDoesNotExist
- exception NonCanonicalProjectBuildings.MultipleObjectsReturned¶
Bases: django.core.exceptions.MultipleObjectsReturned
- NonCanonicalProjectBuildings.objects = <django.db.models.manager.Manager object at 0x7fc70acdda50>¶
- NonCanonicalProjectBuildings.projectbuilding¶
- class seed.models.Project(*args, **kwargs)¶
Bases: django_extensions.db.models.TimeStampedModel
Project(id, created, modified, name, slug, owner_id, last_modified_by_id, super_organization_id, description, status)
- ACTIVE_STATUS = 1¶
- exception DoesNotExist¶
Bases: django.core.exceptions.ObjectDoesNotExist
- Project.INACTIVE_STATUS = 0¶
- exception Project.MultipleObjectsReturned¶
Bases: django.core.exceptions.MultipleObjectsReturned
- Project.STATUS_CHOICES = ((0, <django.utils.functional.__proxy__ object at 0x7fc70ae42d10>), (1, <django.utils.functional.__proxy__ object at 0x7fc70ae42d90>))¶
- Project.adding_buildings_status_percentage_cache_key¶
- Project.building_snapshots¶
- Project.compliance_set¶
- Project.get_compliance()¶
- Project.get_next_by_created(*moreargs, **morekwargs)¶
- Project.get_next_by_modified(*moreargs, **morekwargs)¶
- Project.get_previous_by_created(*moreargs, **morekwargs)¶
- Project.get_previous_by_modified(*moreargs, **morekwargs)¶
- Project.get_status_display(*moreargs, **morekwargs)¶
- Project.has_compliance¶
- Project.last_modified_by¶
- Project.objects = <django.db.models.manager.Manager object at 0x7fc70ae59c50>¶
- Project.owner¶
- Project.project_building_snapshots¶
- Project.removing_buildings_status_percentage_cache_key¶
- Project.super_organization¶
- Project.to_dict()¶
- class seed.models.ProjectBuilding(*args, **kwargs)¶
Bases: django_extensions.db.models.TimeStampedModel
ProjectBuilding(id, created, modified, building_snapshot_id, project_id, compliant, approved_date, approver_id)
- exception DoesNotExist¶
Bases: django.core.exceptions.ObjectDoesNotExist
- exception ProjectBuilding.MultipleObjectsReturned¶
Bases: django.core.exceptions.MultipleObjectsReturned
- ProjectBuilding.approver¶
- ProjectBuilding.building_snapshot¶
- ProjectBuilding.get_next_by_created(*moreargs, **morekwargs)¶
- ProjectBuilding.get_next_by_modified(*moreargs, **morekwargs)¶
- ProjectBuilding.get_previous_by_created(*moreargs, **morekwargs)¶
- ProjectBuilding.get_previous_by_modified(*moreargs, **morekwargs)¶
- ProjectBuilding.noncanonicalprojectbuildings_set¶
- ProjectBuilding.objects = <django.db.models.manager.Manager object at 0x7fc70ae56610>¶
- ProjectBuilding.project¶
- ProjectBuilding.to_dict()¶
- class seed.models.Schema(*args, **kwargs)¶
Bases: django.db.models.base.Model
Groups ColumnMappings together for identification later.
- exception DoesNotExist¶
Bases: django.core.exceptions.ObjectDoesNotExist
- exception Schema.MultipleObjectsReturned¶
Bases: django.core.exceptions.MultipleObjectsReturned
- Schema.columns¶
- Schema.objects = <django.db.models.manager.Manager object at 0x7fc70ad92a10>¶
- Schema.organization¶
- class seed.models.StatusLabel(*args, **kwargs)¶
Bases: django_extensions.db.models.TimeStampedModel
StatusLabel(id, created, modified, name, color, super_organization_id)
- BLUE_CHOICE = 'blue'¶
- COLOR_CHOICES = (('red', <django.utils.functional.__proxy__ object at 0x7fc70ae56490>), ('blue', <django.utils.functional.__proxy__ object at 0x7fc70ae56690>), ('light blue', <django.utils.functional.__proxy__ object at 0x7fc70ae566d0>), ('green', <django.utils.functional.__proxy__ object at 0x7fc70ae56710>), ('white', <django.utils.functional.__proxy__ object at 0x7fc70ae56750>), ('orange', <django.utils.functional.__proxy__ object at 0x7fc70ae56790>), ('gray', <django.utils.functional.__proxy__ object at 0x7fc70ae567d0>))¶
- DEFAULT_LABELS = ['Residential', 'Non-Residential', 'Violation', 'Compliant', 'Missing Data', 'Questionable Report', 'Update Bldg Info', 'Call', 'Email', 'High EUI', 'Low EUI', 'Exempted', 'Extension', 'Change of Ownership']¶
- exception DoesNotExist¶
Bases: django.core.exceptions.ObjectDoesNotExist
- StatusLabel.GRAY_CHOICE = 'gray'¶
- StatusLabel.GREEN_CHOICE = 'green'¶
- StatusLabel.LIGHT_BLUE_CHOICE = 'light blue'¶
- exception StatusLabel.MultipleObjectsReturned¶
Bases: django.core.exceptions.MultipleObjectsReturned
- StatusLabel.ORANGE_CHOICE = 'orange'¶
- StatusLabel.RED_CHOICE = 'red'¶
- StatusLabel.WHITE_CHOICE = 'white'¶
- StatusLabel.canonicalbuilding_set¶
- StatusLabel.get_color_display(*moreargs, **morekwargs)¶
- StatusLabel.get_next_by_created(*moreargs, **morekwargs)¶
- StatusLabel.get_next_by_modified(*moreargs, **morekwargs)¶
- StatusLabel.get_previous_by_created(*moreargs, **morekwargs)¶
- StatusLabel.get_previous_by_modified(*moreargs, **morekwargs)¶
- StatusLabel.objects = <django.db.models.manager.Manager object at 0x7fc70ae63110>¶
- StatusLabel.super_organization¶
- StatusLabel.to_dict()¶
- class seed.models.TimeSeries(*args, **kwargs)¶
Bases: django.db.models.base.Model
For storing engergy use over time.
- exception DoesNotExist¶
Bases: django.core.exceptions.ObjectDoesNotExist
- exception TimeSeries.MultipleObjectsReturned¶
Bases: django.core.exceptions.MultipleObjectsReturned
- TimeSeries.meter¶
- TimeSeries.objects = <django.db.models.manager.Manager object at 0x7fc70acf1490>¶
- class seed.models.Unit(*args, **kwargs)¶
Bases: django.db.models.base.Model
Unit of measure for a Column Value.
- exception DoesNotExist¶
Bases: django.core.exceptions.ObjectDoesNotExist
- exception Unit.MultipleObjectsReturned¶
Bases: django.core.exceptions.MultipleObjectsReturned
- Unit.column_set¶
- Unit.get_unit_type_display(*moreargs, **morekwargs)¶
- Unit.objects = <django.db.models.manager.Manager object at 0x7fc70ae6b4d0>¶
- seed.models.clean_canonicals(b1, b2, new_snapshot)¶
Make sure that we don’t leave dead limbs in our tree.
Parameters: - b1 – BuildingSnapshot, parent 1
- b2 – BuildingSnapshot, parent 2
- new_snapshot – BuildingSnapshot, child.
- seed.models.find_canonical_building_values(org)¶
Get all canonical building snapshots’ id info for an organization.
Parameters: org – Organization inst. Return type: list of tuples, field values specified in BS_VALUES_LIST for all canonical buildings related to an organization. NB: This does not return a queryset!
- seed.models.find_unmatched_buildings(import_file)¶
Get unmatched building snapshots’ id info from an import file.
Parameters: import_file – ImportFile inst. Return type: list of tuples, field values specified in BS_VALUES_LIST. NB: This does not return a queryset!
- seed.models.get_ancestors(building)¶
gets all the non-raw, non-composite ancestors of a building
Recursive function to traverse the tree upward. source_type {
2: ASSESSED_BS, 3: PORTFOLIO_BS, 4: COMPOSITE_BS, 6: GREEN_BUTTON_BS}
Parameters: building – BuildingSnapshot inst. Returns: list of BuildingSnapshot inst., ancestors of building
- seed.models.get_column_mapping(column_raw, organization, attr_name='column_mapped')¶
Callable provided to MCM to return a previously mapped field.
Parameters: - column_raw – str, the column name of the raw data.
- organization – Organization inst.
- attr_name – str, name of attribute on ColumnMapping to pull out. whether we’re looking at a mapping from the perspective of a raw_column (like we do when creating a mapping), or mapped_column, (like when we’re applying that mapping).
Returns: list of mapped items, float representation of confidence.
- seed.models.get_column_mappings(organization)¶
Returns dict of all the column mappings for an Org’s given source type
Parameters: organization – inst, Organization. Returns dict, list of dict: Use this when actually performing mapping between datasources, but only call it after all of the mappings have been saved to the ColumnMapping table.
- seed.models.get_or_create_canonical(b1, b2=None)¶
Gets most trusted Canonical Building.
Parameters: - b1 – BuildingSnapshot model type.
- b2 – BuildingSnapshot model type.
Return type: CanonicalBuilding inst. Will contain PK.
NB: preference is given to existing snapshots’ Canonical link.
- seed.models.get_sourced_attributes(snapshot)¶
Return all the attribute names that get sourced.
- seed.models.initialize_canonical_building(snapshot, user_pk)¶
Called to create a Canonicalbuilding from a single snapshot.
Parameters: - snapshot – BuildingSnapshot inst.
- user_pk – The user id of the user initiating the CanonicalBuilding
- seed.models.obj_to_dict(obj)¶
serializes obj for a JSON friendly version tries to serialize JsonField
- seed.models.save_column_names(bs, mapping=None)¶
Save unique column names for extra_data in this organization.
Basically this is a record of all the extra_data keys we’ve ever seen for a particular organization.
Parameters: bs – BuildingSnapshot instance.
- seed.models.save_snapshot_match(b1_pk, b2_pk, confidence=None, user=None, match_type=None, default_pk=None)¶
Saves a match between two models as a new snapshot; updates Canonical.
Parameters: - b1_pk – int, id for building snapshot.
- b2_pk – int, id for building snapshot.
- confidence – (optional) float, likelihood that two models are linked.
- user – (optional) User inst, last_modified_by for BuildingSnapshot.
Return type: BuildingSnapshot instance, post save.
Determines which Canonical link should be used. If canonical is specified, we’re probably changing a building’s Canonical link, so use that Canonical Building. Otherwise, use the model we match against. If none exists, create it.
Update mapped fields in the new snapshot, update canonical links.
- seed.models.set_initial_sources(snapshot)¶
Sets the PK for the original sources to self.
- seed.models.unmatch_snapshot_tree(building_pk)¶
May or may not obviate unmatch_snapshot. Experimental.
Parameters: building_pk – int - Primary Key for a BuildingSnapshot. Warning
unmatch_snapshot_tree potentially modifies years of merged data. Anything descended from the building_pk will be deleted. The intent is to completely separate building_pk‘s influence on the resultant canonical_snapshot. The user is saying that these are separate entities afterall, yes?
Basically, this function works by getting a merge order list of children from the perspective of building_pk and a list of parents from the perspective of leaf node in the child tree. We take the difference between these lists and call that the remaining_ancestors from which we reconstruct the merge tree for our CanonicalBuilding.
building_pk either gets a reactivated CanonicalBuilding, or a new one.
- seed.models.update_building(old_snapshot, updated_values, user, *args, **kwargs)¶
Creates a new snapshot with updated values.
seed.reconcile module¶
:copyright (c) 2014 - 2015, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Department of Energy) and contributors. All rights reserved. # NOQA :author
- seed.reconcile.build_q(model_a, a_attr, model_b_class, b_attr, a_value=None)¶
build Q objects for model_a in model_b_class among attrs.
Parameters: - model_a – model instance.
- a_attr – str, attribute name on model_a whose value we search for.
- model_b_class – class, the object class we filter through.
- b_attr – str, the attribute name we’re searching.
- seed.reconcile.build_q_filter(q_set, model_a, a_attr, model_b_class, b_attr, op, trans)¶
Build a set of Q objects to filter table results.
Parameters: - q_set – the Q object set that’s being built cumulatively.
- model_a – model instance.
- a_attr – str, the name of an attribute we’re querying from.
- model_b_class – class, model class we’re querying to.
- b_attr – str, model attribute we’re querying to (on model_b_class).
- op – callable, takes two parameters. This callable should be an operator module function, e.g. operator.ior, or operator.iand.
- trans – callable or None. If callable, we apply this callable to
our a_attr for circumstances in which we need to break up its value into sub values for more accurate querying (e.g. address data.).
- seed.reconcile.calculate_confidence(model_a, model_b, mapping='FIRST_PASS')¶
Determine the similarity between model_a and model_b.
Goes through the mappings and compares those attrs between each of the modules produced in the search function.
Rtype float: 0.0 to 1.0, the degree that the two models are similar.
- seed.reconcile.get_best_match(model_a, search_results)¶
Return highest confidence model_b match and its confidence number.
Parameters: - model_a – model instance, the known model.
- search_results – a queryset of model_b insts returned from search
Rtype tuple: (float, model_b_inst|None)
- seed.reconcile.logger = <celery.utils.log.ProcessAwareLogger object at 0x7fc706a06190>¶
Module seeks to provide a general way to tie together information about a single entity from two datasources using sparse data.
Sample usage:
A task or a view that wants to find a correlated building from another data set would execute the following set of calls:
- model_a is an instance of a model that I have (probably just recently
- loaded).
model_b_qs is the dataset type we’re trying to find matches in.
` possible_matches = search(model_a, model_b_qs) best_match = get_best_match(model_a, possible_matches) best_match >>> (0.89, AssessedBuildingInst) `
- seed.reconcile.search(model_a, model_b_qs, mapping='FIRST_PASS', op=<built-in function ior>)¶
Create and execute filters according to mapping rules.
Parameters: - model_a – model instance we’re querying from.
- model_b_qs – QuerySet, the pre-filtered set of models to match.
- mapping – str, prefix for a mapping between model_a and model_b.
- op – callable, how we combine this set of Q objects.
Rtype Queryset: the Q objects are strung together and passed filter. The results, if any, are the set of possible matches.
seed.search module¶
:copyright (c) 2014 - 2015, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Department of Energy) and contributors. All rights reserved. # NOQA :author
- seed.search.build_json_params(order_by, sort_reverse)¶
returns db_columns, extra_data_sort, and updated order_by
Parameters: order_by (str) – field to order_by Returns: tuple: db_columns: dict of known DB columns i.e. non-JSONfFeld, extra_data_sort bool if order_by is in extra_data JsonField, order_by str if sort_reverse and DB column prepend a ‘-‘ for the django order_by
returns a list of sibling and parent orgs
- seed.search.convert_to_js_timestamp(timestamp)¶
converts a django/python datetime object to milliseconds since epoch duplicated code with seed utils due to circular imports
- seed.search.create_building_queryset(orgs, exclude, order_by, other_orgs=None, extra_data_sort=False)¶
creates a querset of buildings within orgs. If other_orgs, buildings in both orgs and other_orgs will be represented in the queryset.
Parameters: - orgs – queryset of Organization inst.
- exclude – django query exclude dict.
- order_by – django query order_by str.
- other_orgs – list of other orgs to or the query
- seed.search.filter_other_params(queryset, other_params, db_columns)¶
applies a dictionary filter to the query set. Does some domain specific parsing,
mostly to remove extra query params and deal with ranges. Ranges should be passed in as ‘<field name>__lte’ or ‘<field name>__gte’
e.g. other_params = {‘gross_floor_area__lte’: 50000}Parameters: - Queryset queryset (Django) – queryset to be filtered
- other_params (dict) – dictionary to be parsed and applied to filter.
- db_columns (dict) – list of column names, extra_data blob outsite these
Returns: Django Queryset:
- seed.search.generate_paginated_results(queryset, number_per_page=25, page=1, whitelist_orgs=None, below_threshold=False)¶
- returns a page of results as a list from the queryset for the given
- fields
- Usage:
- generate_paginated_results(q, 1)
Out: [{
‘gross_floor_area’: 1710, ‘site_eui’: 123, ‘tax_lot_id’: u’IMP9-test001’, ‘year_built’: 2001}]
Parameters: - queryset – optional queryset to filter from
- number_per_page (int) – optional number of results per page
- page (int) – optional page of results to get
- whitelist_orgs – a queryset returning the organizations in which all building fields can be returned, otherwise only the parent organization’s exportable_fields should be returned. The whitelist_orgs are the orgs the request user belongs.
- below_threshold – True if less than the parent org’s query threshold is greater than the number of queryset results. If True, only return buildings within whitelist_orgs.
- seed.search.get_building_fieldnames()¶
returns a list of field names for the BuildingSnapshot class/model that will be searched against
- seed.search.get_orgs_w_public_fields()¶
returns a list of orgs that have publicly shared fields
- seed.search.is_not_whitelist_building(parent_org, building, whitelist_orgs)¶
returns false if a building is part of the whitelist_orgs
Parameters: - parent_org – the umbrella parent Organization inst.
- building – the BuildingSnapshot inst.
- whitelist_orgs – queryset of Organization insts.
Returns: bool
- seed.search.mask_results(search_results)¶
masks (deletes dict keys) for non-shared public fields
- seed.search.orchestrate_search_filter_sort(params, user)¶
Given a parsed set of params, perform the search, filter, and sort for BuildingSnapshot’s
- seed.search.paginate_results(request, search_results)¶
returns a paginated list of dict results
- seed.search.parse_body(request)¶
parses the request body for search params, q, etc
Params request: django wsgi request object Returns: dict { ‘exclude’: dict, exclude dict for django queryset ‘order_by’: str, query order_by, defaults to ‘tax_lot_id’ ‘sort_reverse’: bool, True if ASC, False if DSC ‘page’: int, pagination page ‘number_per_page’: int, number per pagination page ‘show_shared_buildings’: bool, whether to search across all user’s orgs ‘q’: str, global search param ‘other_search_params’: dict, filter params ‘project_id’: str, project id if exists in body }
- seed.search.process_search_params(params, user, is_api_request=False)¶
Given a python representation of a search query, process it into the internal format that is used for searching, filtering, sorting, and pagination.
Parameters: - params – a python object representing the search query
- user – the user this search is for
- is_api_request – bool, boolean whether this search is being done as an api request.
Returns: dict { ‘exclude’: dict, exclude dict for django queryset ‘order_by’: str, query order_by, defaults to ‘tax_lot_id’ ‘sort_reverse’: bool, True if ASC, False if DSC ‘page’: int, pagination page ‘number_per_page’: int, number per pagination page ‘show_shared_buildings’: bool, whether to search across all user’s orgs ‘q’: str, global search param ‘other_search_params’: dict, filter params ‘project_id’: str, project id if exists in body
}
- seed.search.remove_results_below_q_threshold(search_results)¶
removes buildings if total count of buildings grouped by org is less than their org’s public query threshold
Parameters: search_results (list/queryset) – search results Returns: list or queryset
- seed.search.search_buildings(q, fieldnames=None, queryset=None)¶
returns a queryset for matching buildings
Parameters: - or unicode q (str) – search string
- fieldnames (list) – list of BuildingSnapshot model fieldnames (defaults to those generated by get_building_field_names())
- queryset – optional queryset to filter from, defaults to BuildingSnapshot.objects.none()
Returns: queryset: queryset of matching buildings
- seed.search.search_public_buildings(request, orgs)¶
- returns a queryset or list of buildings matching the search params and
- count
Parameters: request – wsgi request (Django) for parsing params Orgs: list of Organization instances to search within Returns: tuple (search_results_list, result count)
seed.tasks module¶
:copyright (c) 2014 - 2015, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Department of Energy) and contributors. All rights reserved. # NOQA :author
- exception seed.tasks.DuplicateDataError(id)¶
Bases: exceptions.RuntimeError
- seed.tasks.add_cache_increment_parameter(tasks)¶
This adds the cache increment value to the signature to each subtask.
- seed.tasks.apply_data_func(mappable_columns)¶
Returns a function that captures mappable_types in a closure and will add a key to extra data if not in mappable_types else
- seed.tasks.apply_extra_data(model, key, value)¶
Function sent to MCM to apply mapped columns into extra_data.
- seed.tasks.cache_first_rows(import_file, parser)¶
Cache headers, and rows 2-6 for validation/viewing.
Parameters: - import_file – ImportFile inst.
- parser – unicode-csv.Reader instance.
Unfortunately, this is duplicated logic from data_importer, but since data_importer makes many faulty assumptions we need to do it differently.
- seed.tasks.get_canonical_id_matches(org_id, pm_id, tax_id, custom_id)¶
Returns canonical snapshots that match at least one id.
- seed.tasks.get_canonical_snapshots(org_id)¶
Return all of the BuildingSnapshots that are canonical for an org.
- seed.tasks.handle_id_matches(unmatched_bs, import_file, user_pk)¶
“Deals with exact matches in the IDs of buildings.
- seed.tasks.handle_results(results, b_idx, can_rev_idx, unmatched_list, user_pk)¶
Seek IDs and save our snapshot match.
Parameters: - results – list of tuples. [(‘match’, 0.99999),...]
- b_idx – int, the index of the current building in the unmatched_list.
- can_rev_idx – dict, reverse index from match -> canonical PK.
- user_pk – user ID, used for AuditLog logging
Unmatched_list: list of dicts, the result of a values_list query for unmatched BSes.
- seed.tasks.is_same_snapshot(s1, s2)¶
seed.token_generators module¶
:copyright (c) 2014 - 2015, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Department of Energy) and contributors. All rights reserved. # NOQA :author
- class seed.token_generators.SignupTokenGenerator¶
Bases: object
Strategy object used to generate and check tokens for the password reset mechanism.
- check_token(user, token, token_expires=True)¶
Check that a password reset token is correct for a given user.
- make_token(user)¶
Returns a token that can be used once to do a password reset for the given user.
seed.urls module¶
:copyright (c) 2014 - 2015, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Department of Energy) and contributors. All rights reserved. # NOQA :author
seed.utils module¶
:copyright (c) 2014 - 2015, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Department of Energy) and contributors. All rights reserved. # NOQA :author
seed.views module¶
:copyright (c) 2014 - 2015, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Department of Energy) and contributors. All rights reserved. # NOQA :author
Module contents¶
:copyright (c) 2014 - 2015, The Regents of the University of California, through Lawrence Berkeley National Laboratory (subject to receipt of any required approvals from the U.S. Department of Energy) and contributors. All rights reserved. # NOQA :author