db_tools package
Modules
There are two essential database scripts. The first is to write a set of influxDB line protocol text files that can be uploaded to the oTherm database, and the second is a set of functions that use API requests to retrieve and store data into local pandas.DataFrame and dataclass objects.
db_tools.influx_lp_writer module
db_tools.otherm_db_reader module
A collection of functions that use oTherm APIs to retrieve data from an oTherm instance. The typical application is to first retrieve the site data. Then, using the site dataclass object, retrieve information about the:
weather_station,
thermal_load,
monitoring_system, and
heat_pump_data.
The tools also contain scripts for:
Retrieving the specifications for any oTherm monitoring system by the name of the monitoring system, and
Retrieving heat pump peformance data from a local SQLite database (note, the SQLite database is not part of the oTherm database.
Note
The names and types of data elements used in the analyses differ from the oTherm data model specification.
The dataclass objects use for analysis are constructed from json objects returned from the oTherm database. However, because the dataclass objects represent a single instance, the data elements are reorganized into a simpler representation than the original json response.
Example
The input typically consists of a site_name and start and end dates. The functions can be called from analyses modules. For example
site_name = 'GES649'
start_date = '2015-01-01'
end_date = '2021-01-01'
#Get site information
site = get_site_info(site_name)
#Get equipment information and dataframe of heat pump operating data
equipment, hp_data = get_equipment_data(site.id, start_date, end_date, site.timezone)
#Get monitoring system information and measurement specifications
equip_monitoring_system = get_equipment_monitoring_system(equipment.id)
#Get weather data for station
wx_data = get_weather_data(site.weather_station.nws_id, site.timezone, start_date, end_date)
#Get thermal source specifications
source_specs = get_source_specs(site)
- db_tools.otherm_db_reader.get_site_info(site_name, db)
get site info docstring
- Parameters:
site_name (str) – name of oTherm site
- Returns:
The site object consists is a nested dataclass object
@dataclass class Site: id: int name: str city: str state: str timezone: str thermal_load: ThermalLoad weather_station: WeatherStation
To access data elements, use the dot syntax. For example, the Weather Station ID, is accessed by
>>> site.weather_station 'KPSM'
- db_tools.otherm_db_reader.get_thermal_load(site, db)
- Dataclass object with equipment specifications ::
@dataclass class ThermalLoad:
uuid: str name: str description: Optional[str] conditioned_area: float heating_design_load: float cooling_design_load: float heating_design_oat: float cooling_design_oat: float
To access data elements, use the dot syntax. For example, the Weather Station ID, is accessed by
- db_tools.otherm_db_reader.get_equipment(site_id, db)
Uses ‘request’ method to read equipment table for a specific site
- Parameters:
site_id (int) – The site_id in the PostgreSQL database. Can be obtained from site.id
- Returns:
Equipment dataclass contains equipment information in the following fields
@dataclass
- class Equipment:
id: int uuid: str model: str description: Optional[str] no_flowmeter_flowrate: float type: int site: int manufacturer: int
- db_tools.otherm_db_reader.get_equipment_data(site_id, start_date, end_date, timezone, db)
Uses ‘request’ method to reads heat pump operating data from otherm influx database and returns a pandas dataframe. The data DataFrame returned includes all records for the equipment at a site. At present, the script is limited to a single piece of equipment at a site.
- Parameters:
site_id (int) – The site_id in the PostgreSQL database. Can be obtained from site.id
start_date (str) – start date (e.g. 2018-1-1)
end_date (str) – end date (e.g. 2018-12-31)
timezone (str) – (e.g. ‘US/Eastern’)
- Returns:
Equipment dataclass contains equipment information in the following fields:
@dataclass class Equipment: id: int uuid: str model: str description: Optional[str] no_flowmeter_flowrate: float type: int site: int manufacturer: int
pandas.DataFrame containing heat pump operating data over the specified time range. The DataFrame contains all fields stored for the piece of equipment in the influxDB database.
Note
The index of the DataFrame is set to the
time
field and localized according thesite.timezone
attribute
- db_tools.otherm_db_reader.get_equipment_monitoring_system(equip_id)
Retrieves the equipment monitoring system and specifications
- Parameters:
uuid (str) – uuid of thermal equipment
- Returns:
Dataclass object with equipment monitoring system specifications
@dataclass class MonitoringSysInfo: id: int name: Optional[str] description: Optional[str] specs: list @dataclass class EquipmentMonitor: id: int start_date: str end_date: Optional[str] equip_id: int monitoring_system_spec: int info: MonitoringSysInfo
To access data elements, use the dot syntax. For example, the list containing the monitoring system specifications can be accessed by
>>> monitoring_system.info.specs `[{'measurement_spec': {'name': 'HPP VA W 8% EP', 'description': 'Heat pump power, volt-amps, electrical panel', ...`
The monitoring system specifications is a list of measurements performed by the monitoring system, each measurement has its own set of specifications. See oTherm documentation for more details.
The list can be search for individual measurements specifications with
utilities.get_measurement_specs
- db_tools.otherm_db_reader.get_weather_data(nws_id, timezone, start_date, end_date)
- Parameters:
nws_id (str) – National Weather Station 4 character station identifier
timezone (str) – Timezone of site, such as ‘US/Eastern’
start_date (str) – Beginning date of request, such as ‘2015-01-01’
end_date (str) – End date of request
- Returns:
- pandas.DataFrame
The returned DataFrame contains weather station data over the specified time range and contains all fields stored for the weather station.
Note
The index of the DataFrame is set to the
time
field and localized according thesite.timezone
attribute
- db_tools.otherm_db_reader.get_source_specs(site)
Retrieves the source specifications.
- Parameters:
site (str) – site name
- Returns:
Dataclass object with source specifications
@dataclass class SourceSpec: site: str site_id: int source_name: str source_type: str description: str freeze_protection: Optional[float] grout_type: Optional[str] formation_conductivity: Optional[float] formation_type: Optional[str] grout_conductivity: Optional[float] antifreeze: Optional[str] pipe_dimension_ratio: Optional[str] n_pipes_in_circuit: Optional[int] n_circuits: Optional[int] total_pipe_length: Optional[float]
To access data elements, use the dot syntax.
Note
While the oTherm data model supports multiple types of sources, this db_reader tool only supports the vertical loop spec at present.
- db_tools.otherm_db_reader.get_mfr_data(parameters)
- db_tools.otherm_db_reader.get_monitoring_system(name)
Similar to
get_equipment_monitoring_system()
but returns monitoring_system attributes for a given monitoring system by name rather than equipment being monitored. This function requires the exact name of the monitoring system, as specified in the oTherm database- Parameters:
name (str) – The name of the monitoring system
- Returns:
All specifications of a monitoring system in the oTherm database. Refer to oTherm documentation for detais.
- Return type:
dict
For more explanation of the parameters and return values, see
get_equipment_monitoring_system()
db_tools.csv_to_yaml module
- db_tools.csv_to_yaml.output_yaml(equipment_model, site_model, thermal_load_model)