Introduction
Snowflake is a cloud-native data platform with storage and features to work with structured, semi-structured, and unstructured data.
As the All-Data, Any-AI Platform, FME provides a valuable extension to Snowflake. FME can load and process any type of data in Snowflake that would not otherwise be possible without writing and maintaining code.
For example:
- Transform and load any type of data into Snowflake
- Interrogate your Snowflake data with any AI
- Join unstructured, semi-structured, and structured data together to create a complete dataset for analysis
- Perform additional validation, processing, and analysis on spatial data
- Stream data from IoT sensors into your data lakehouse using the KafkaConnector, MQTTConnector, Websockets, and more
- Run the FME engine directly inside Snowflake, putting the processing beside your data for improved performance and security
This article provides links to resources to help get you started with Snowflake and FME.
Connecting to Snowflake
Before you can read or write Snowflake data, you first need to get connected to your Snowflake account. You should have access to a Snowflake account URL, likehttps://<your-company>.snowflakecomputing.com. If you don't have a Snowflake account yet, you can signup or contact Snowflake sales.
FME offers several different ways to connect to Snowflake. All connection methods use Snowflake's JDBC driver. Follow the steps in one of the articles below for the method you choose.
- How to Authenticate with Snowflake OAuth (Custom Client) in FME
- How to Authenticate with Snowflake: External OAuth with Azure AD
- Connecting to Snowflake using Keypair Authentication
If you have issues with your Snowflake connection, please see Troubleshooting Connections to Snowflake Database.
Work with Snowflake Data in FME
Once you're successfully connected, you can start using FME to work with data in Snowflake.
The article below introduces the Snowflake Reader and Writer, as well as techniques to use your Snowflake connection in FME's SQL transformers, which lets you use Snowflake SQL functions.
To learn more about building dynamic JSON and working with Snowflake Variants in FME, see:
Advanced Snowflake and FME Techniques
You can use the SQLCreator and SQLExecutor to interact with your Snowflake database for finer control over staging and more.
- Use a PUT command to upload a supported file type, like CSV or Parquet, to an internal stage. Load the file data into an existing database table using a COPY INTO command.
- Use FME to send files to external stages in Amazon S3, Google Cloud Storage, or Microsoft Azure. Load the file data into an existing database table using a COPY INTO command.
- Run a Snowflake MERGE command against both new data and existing data, performing all necessary insert, update, and delete operations.
- Create tables, views, and other Snowflake objects.
Do anything you can do with Snowflake SQL inside FME alongside all the functionality offered by hundreds of transformers and formats not supported by Snowflake.
FME Remote Engines Services in Snowflake
FME Engines are what process a translation. You can parallelize your workflows and run them beside your Snowflake data for increased security and performance using FME Remote Engines Services in Snowflake. To learn more, please see the article below.
Additional Resources