Introduction
Google BigQuery is an enterprise data warehouse that enables users to run SQL-like queries against extremely large-scale data in a matter of seconds. BigQuery supports both business and spatial data.
FME can help get business and spatial data into BigQuery, where users can leverage analysis services on their data. Once support for writing to Google BigQuery was added in FME, users could easily migrate business and spatial data from formats supported by FME into BigQuery.
This tutorial series will provide an introduction to working with the Google BigQuery format within FME.
Requirements
- Google BigQuery Account
- Granted the required permissions to create tables in BigQuery
Terminology
Project:
Google Cloud projects serve as the foundation for creating, enabling, and utilizing all Google Cloud services. Users will need to have access to a Google Cloud Project and be granted the necessary permissions to connect to BigQuery.
Dataset:
BigQuery datasets are top-level containers for tables and views within a specific project. They are used to organize and control access to tables and views. FME is currently not able to create new datasets, so tables must be written to existing datasets.
Note that the BigQuery concept of a dataset is slightly different than FME's concept of a dataset. BigQuery's definition of a dataset is similar to a database schema in a relational database or a Feature Dataset in the File Geodatabase format.
Table:
BigQuery tables contain individual records. Each table is defined by a schema that describes column names, data types, and other information. A BigQuery table is analogous to a feature type in FME.