This is the fourth in a six part series which will provide details on a solution I have been working on which will gather information from a variety of sources to forecast how much electricity will be generated by a residential solar panel system. This example shows how OMS can be used to gather data from multiple sources, how to visualize this data in Power BI and how to use a variety of data sources in OMS to forecast likely future results. This series includes the following posts:
- Gathering weather data into Microsoft OMS (the script details are available here)
- Using Power BI to visualize weather data from Microsoft OMS
- How to add weather forecasting information into Microsoft OMS
- How to add SolarCity electricity generation information into Microsoft OMS
- How to use Power BI to cross multiple custom log types
- How to bring this all together and forecast electricity generation
In this blog post we will see an example of how information can be gathered from the SolarCity API and added to Microsoft OMS.
Getting the SolarCity data:
SolarCity provides an API which is available here. They also have a sample Python 2.7 script which I built from which is available here. Microsoft provides a Python 2.7 example to get data into OMS using the ingestion API which is available here.
For my environment, I created a virtual running in Azure (Standard A1 for about $32 a month) where I installed the required software and took the following steps to get the script to run:
- Downloaded & installed Python 2.7.12
- Downloaded & installed Pip
- Downloaded & imported Requests (and all other modules)
- Test script
- Scheduled the script to run in the Windows task scheduler.
TIP: There are several fields that you need to provide for this script, details on where to get them are provided below:
- USERNAME (SolarCity user name)
- PASSWORD (SolarCity user password)
- CLIENTSECRET (Contact SolarCity for your Clientsecret)
- OMSWORKSPACE (OMS workspace name, available in Log Analytics, settings)
- OMSWORKSPACEKEY (OMS primary key, available in Log Analytics, settings)
NOTE: In the future I expect that we will be able to run Python scripts like this without having to run a full virtual machine potentially in either Function Apps or as part of Azure Automation but for now this was the easiest way to get the script to run on a scheduled basis.
The Python script and the solution file for Log Analytics are available for download here.
Using the SolarCity data in Log Analytics:
Once data has been sent to the Log Analytics workspace, we can import the OMS solution to visualize the data (this solution file is available for download with the Python script). The examples below show the top level dashboard view and the drill down which shows more details on kWh generated on daily and 15 minute intervals.
NOTE: This solution is designed for the new Log Analytics query language so if you want to use this solution you will need to upgrade your workspace to the new query language if you haven’t done so already.
The data can be queried by the following query for the custom class which we created (SolarCity): SolarCity_CL
Summary: In this blog post we have shown how you can use a Python 2.7 script to gather data from a SolarCity API and write this information into Log Analytics. Once this data is in place we can query and visualize the data as shown above. In the next part of this series we will show how you can use Power BI to query across several custom classes that were created in this series.