Capabilities
The following enhanced BigQuery capabilities enabled by the flowfunctions library are currently available to be deployed onto client projects:
Inbound Data
These capabilities give insight into the existence and metadata relating to inbound data, and are typically required when using non-Google-native data ingestion methods. However for Google-native sources it is often useful to monitor ingest times as these can be unpredictable, potentially impacting downstream activities.
| Capability |
|---|
| Inbound Data Ingest Time Tracking |
| Inbound Date Partition Profiling |
| Inbound Data Profiling |
| Inbound Data Monitoring (Slack Notification) |
| Inbound Data Exception Notification (Slack Notification) |
| Inbound Event-based Function Triggering |
Data Profiling
These capabilities give insights into the contents of data at any point in a data workflow.
| Capability |
|---|
| Partition Comparison Analysis |
| Data Duplicate Flagging |
| Data Distribution Profiling |
| Data Similarity Analysis |
| Nested Column Profiling |
| Schema Evolution Tracking |
Data Monitioring
These capabilities allow passive or active monitoring of data based on defined rules and expectations.
| Capability |
|---|
| Passive Data Monitoring (Dashboard) |
| Active Data Monitoring (Notifications) |
Resource Profiling
These capabilities give insight into resources such as datasets, tables, views and scheduled queries.
| Capability |
|---|
| Resource Validity Analysis |
| Obsolete Resource Identification |
| Scheduled Query Inventory |
| Data Transfer Job Inventory |
| Data Transfer Job History |
| Object Table Deployment |
Depency & Lineage
These capabilities analyse resource metadata to give insight into relationships between resources.
| Capability |
|---|
| Data Dependency Mapping |
| Scheduled Query Dependency Mapping |
| Data Dependency Visualisation |
| Data Lineage Mapping |
| Column Lineage Mapping |
| Column Dependency Mapping |
Usage Analysis
These capabilities give insight into usage and cost of BigQuery and Google Cloud resources.
| Capability |
|---|
| Query Analysis |
| Billing Analysis |
| Looker Studio Usage Analysis |
Data Management
These capabilities give insight and enable automation of data management activities.
| Capability |
|---|
| Data Backup Automation |
| Data Restore |
| SQL Backup Automation |
| IAM Policy Analysis |
| IAM Policy Configuration |
| GCS Object Signed URL Generation |
Google Analytics
These capabilities help solve common Google Analytics challenges, predominatly for GA4.
| Capability |
|---|
| GA4 Nested Column Profiling |
| GA4 Property & Dimension Decoding |
| GA4 BigQuery Automation |
Google Cloud APIs
These capabilities give access to Google Cloud APIs from within BigQuery data workflows, using Remote Functions connected to Python Cloud Functions and Secret Manager for security.
| Capability |
|---|
| Looker Studio Linking API |
| Google Cloud Storage |
| Google Natural Language |
| Google Translate |
| Google Document AI |
External APIs
These capabilities extend BigQuery by accessing external APIs from within BigQuery data workflows, using Remote Functions connected to Python Cloud Functions and Secret Manager for security.
| Capability |
|---|
| GPT4 Integration |
| Addition Custom Integration on Demand |