Skip to main content

Posts

Showing posts with the label BigQuery

Google Cloud IoTCore with UDOO Quad

UDOO Quad is a maker SBC board from UDOO . It has a built-in Arduino microcontroller and can run Linux on the SBC. On the web, most of the Google IoT Core samples have been focused using Mongoose OS and I have written a post on " Journey into Google Cloud IoT Core with ESP8266 and Mongoose OS ". Recently, Google releases code samples which make the initial handshaking with Cloud IoTCore simpler.  For UDOO, the tedious part is to get all supporting tools to work with a particular version of Linux Distribution. UDOO supports a few Linux distributions (Ubuntu, Debian and others). I have installed Ubuntu and the current version supported by UDOO is 14.04 LTS. For this demo, I wanted to work on Python 3.5 and I have compiled the Python from source. This is step may not be necessary if you use the version that is installed (Python 3.4). The idea for this project is to read the temperature and humidity reading from UDOO Arduino side, transfer the data to Linux side and forward t...

Import CSV file from Google Drive to Big Query

I have written about using  App Scripts to collect telemetry data  and the ease of integrating it to Google Sheet to chart the temperature and humidity reading. In this post, I shall describe the approach using a CSV uploaded via Google File Stream and a scheduled job which is written in App Script to load the data into Big Query. This approach is very good for integrating the transactional capability of ERP system and the Analytics capability of Visualisation system (Data Studio). ERP system has rich built-in business process workflow and transactional control, When performing Dashboard and Data Analytics functionality, it will slow down the entire system. In this case, it may be better for analytics features to be performed by another system. Fortunately, most ERP system is capable of auto execute a query to export CSV data. I have made use of this capability to import the daily files generated by ERP to Big Query via Google Drive. This CSV file is picked up by an App Scri...