Snowflake Notebook Migration with CI/CD: A Structured Approach

In today’s data-driven enterprises, ensuring consistency, governance, and automation in the development and deployment of analytical assets is critical. As organizations scale their Snowflake adoption, managing notebooks through traditional manual methods becomes inefficient and error-prone. To address this, integrating notebooks with Git-backed repositories and a robust CICD pipeline becomes essential.

This blog outlines a structured and secure approach to migrating and deploying Snowflake notebooks using GitHub and Snowflake’s native integration capabilities bringing modern software engineering practices to your data workflows.

Step 1: Create an API Integration with GitHub

Snowflake allows direct integration with GitHub via the CREATE API INTEGRATION command. This integration authorizes Snowflake to access a GitHub repository securely.

CREATE OR REPLACE API INTEGRATION git_api_integration

  API_PROVIDER = git_https_api

  API_ALLOWED_PREFIXES = (‘https://github.com/’)

  ENABLED = TRUE;

Step 2: Create Databases and Git Repositories

Create the required databases and schemas to host and manage our notebooks.

CREATE DATABASE DEVOPS;

CREATE SCHEMA COMMON;

Create secret for your PAT TOKEN if you are using a private repository.

CREATE OR REPLACE SECRET GITHUB_NB_SECRET

  TYPE = password

  USERNAME = ‘<your_username>’

  PASSWORD = ‘<your_password>’;

Create an API integration that specifies details for the Snowflake interaction with the Git repository API

CREATE OR REPLACE API INTEGRATION git_api_integration

API_PROVIDER = git_https_api

API_ALLOWED_PREFIXES = (‘https://github.com/’)

ALLOWED_AUTHENTICATION_SECRETS = (‘GITHUB_NB_SECRET’)

ENABLED = TRUE;

Connect GitHub repositories as Git-backed repositories inside Snowflake:

CREATE OR REPLACE GIT REPOSITORY devops.common.git_repo_notebook

  API_INTEGRATION = git_api_integration

  ORIGIN = ‘https://github.com/Darshini26/snowflakenotebook’;

Step 3: Set Up User and Roles

Create a dedicated user for Git-based notebook access and assign granular privileges through a custom role.

CREATE USER GIT_USER;

ALTER USER GIT_USER SET RSA_PUBLIC_KEY = ‘<your-public-key>’;

CREATE ROLE GIT_USER_ROLE;

GRANT ROLE GIT_USER_ROLE TO USER GIT_USER;

GRANT USAGE ON WAREHOUSE COMPUTE_WH TO ROLE GIT_USER_ROLE;

GRANT USAGE ON DATABASE DEVOPS TO ROLE GIT_USER_ROLE;

GRANT USAGE ON SCHEMA DEVOPS.COMMON TO ROLE GIT_USER_ROLE;

GRANT READ, WRITE ON GIT REPOSITORY DEVOPS.COMMON.GIT_REPO_NOTEBOOK TO ROLE GIT_USER_ROLE;

GRANT CREATE NOTEBOOK ON SCHEMA DEV.PUBLIC TO ROLE GIT_USER_ROLE;

GRANT CREATE NOTEBOOK ON SCHEMA PROD.PUBLIC TO ROLE GIT_USER_ROLE;

GRANT ROLE GIT_USER_ROLE TO ROLE ACCOUNTADMIN;

Step 4: Generate and Configure RSA Keys

Use OpenSSL to generate secure RSA keys for authenticating the Git user:

bash

# Generate private key

openssl genpkey -algorithm RSA -out snowflake_rsa_key.p8 -pkeyopt rsa_keygen_bits:2048

# Extract public key

openssl rsa -in snowflake_rsa_key.p8 -pubout -out snowflake_rsa_key.pub

# Convert to PKCS8 format

openssl pkcs8 -topk8 -nocrypt -in snowflake_rsa_key.p8 -out new_snowflake_rsa_key.p8

Step 5: Configure Repository Secrets in GitHub

To enable secure authentication with Snowflake, create the following secrets in your GitHub repository under Settings → Secrets and variables → Actions → Repository secrets:

• SNOWFLAKE_ACCOUNT
• SNOWFLAKE_USER
• SNOWFLAKE_PASSWORD (if using password-based auth)
  or
  SNOWFLAKE_PRIVATE_KEY (if using JWT-based auth)

These secrets will be referenced securely in the GitHub Actions workflow.

Step 6: Sample CICD YAML Configuration

GitHub Actions: Snowflake Notebook Deployment

Please find the GitHub Actions workflow to deploy notebooks to Snowflake automatically:

https://github.com/Darshini26/Snowflakenotebook_1/blob/main/notebooks.yml

This workflow automates notebook deployment to Snowflake.

• Triggers on push to dev or manual dispatch.
• Deploys to Dev: Sets up Snowflake CLI, configures auth, fetches repo, and deploys notebook.
• Manual Approval: Required before production deployment.
• Merges Dev to Prod after approval.
• Deploys to Prod with similar steps targeting the production environment.

Conclusion

By integrating GitHub with Snowflake for notebook development, you bring version control, review processes, and CICD discipline into your data workflows. This migration empowers teams to work collaboratively, deliver faster, and maintain strong governance over analytical assets.

Please feel free to reach out to us for your Snowflake solution needs. Cittabase is a Premier  partner with Snowflake.