The following enhanced BigQuery capabilities enabled by the
flowfunctions library are currently available to be deployed onto client projects:
These capabilities give insight into the existence and metadata relating to inbound data, and are typically required when using non-Google-native data ingestion methods. However for Google-native sources it is often useful to monitor ingest times as these can be unpredictable, potentially impacting downstream activities.
|Inbound Data Ingest Time Tracking|
|Inbound Date Partition Profiling|
|Inbound Data Profiling|
|Inbound Data Monitoring (Slack Notification)|
|Inbound Data Exception Notification (Slack Notification)|
|Inbound Event-based Function Triggering|
These capabilities give insights into the contents of data at any point in a data workflow.
|Partition Comparison Analysis|
|Data Duplicate Flagging|
|Data Distribution Profiling|
|Data Similarity Analysis|
|Nested Column Profiling|
|Schema Evolution Tracking|
These capabilities allow passive or active monitoring of data based on defined rules and expectations.
|Passive Data Monitoring (Dashboard)|
|Active Data Monitoring (Notifications)|
These capabilities give insight into resources such as datasets, tables, views and scheduled queries.
|Resource Validity Analysis|
|Obsolete Resource Identification|
|Scheduled Query Inventory|
|Data Transfer Job Inventory|
|Data Transfer Job History|
|Object Table Deployment|
Depency & Lineage
These capabilities analyse resource metadata to give insight into relationships between resources.
|Data Dependency Mapping|
|Scheduled Query Dependency Mapping|
|Data Dependency Visualisation|
|Data Lineage Mapping|
|Column Lineage Mapping|
|Column Dependency Mapping|
These capabilities give insight into usage and cost of BigQuery and Google Cloud resources.
|Looker Studio Usage Analysis|
These capabilities give insight and enable automation of data management activities.
|Data Backup Automation|
|SQL Backup Automation|
|IAM Policy Analysis|
|IAM Policy Configuration|
|GCS Object Signed URL Generation|
These capabilities help solve common Google Analytics challenges, predominatly for GA4.
|GA4 Nested Column Profiling|
|GA4 Property & Dimension Decoding|
|GA4 BigQuery Automation|
Google Cloud APIs
These capabilities give access to Google Cloud APIs from within BigQuery data workflows, using Remote Functions connected to Python Cloud Functions and Secret Manager for security.
|Looker Studio Linking API|
|Google Cloud Storage|
|Google Natural Language|
|Google Document AI|
These capabilities extend BigQuery by accessing external APIs from within BigQuery data workflows, using Remote Functions connected to Python Cloud Functions and Secret Manager for security.
|Addition Custom Integration on Demand|