Hi everyone, our dashboards currently rely on full data loads. However, we’d like some help setting them up to use only incremental loads so we don't put a strain on our database.

Thinking completed In GoodData Cloud, analytics are executed live against your underlying data source—GoodData does not ingest or duplicate your data via internal ETL [Community Slack]. Instead, dashboards generate standard SQL queries that execute directly in your database to compute visualizations [Improve Database Performance; Community Slack]. Because GoodData does not poll or store raw database data incrementally for dashboards, configuring “incremental loads” to reduce database strain is primarily handled at your database design and caching layer: 1. Database-Level Optimizations To prevent heavy full-table scans when GoodData queries run: • Materialized Views: Many modern databases support incrementally updated materialized views. You can pre-aggregate, pre-join, or incrementally update transformed views in your database so GoodData queries read from lightweight, prepared tables rather than raw data [Improve Database Performance]. • Table Partitioning: Partition large fact tables (such as by date granularity). This allows the database engine to use partition pruning when dashboard filters (like date ranges) are applied, scanning only the relevant slice of data [Improve Database Performance]. • Denormalization and Columnar Storage: Utilize columnar formats and wider denormalized tables so that queries only read the specific columns requested by the visualization [Improve Database Performance]. 2. Caching Strategy (FlexCache) • GoodData uses FlexCache to store raw and final query results, evicting entries via an LRU (Least Recently Used) mechanism [Community Slack; Community Slack]. When a query result is cached, users are served immediately without querying your database. • Once your data pipeline finishes updating your database tables, trigger the cache invalidation endpoint:POST /api/v1/actions/dataSources/{dataSourceId}/uploadNotification[Community Slack] • You can optionally pre-warm the cache after an update by executing AFM queries for key dashboards via

POST /execution/afm/execute

, reducing subsequent load on the database when end users open their dashboards [Community Slack]. _(Note: If by “loads” you were referring to entity provisioning, such as users, user groups, or workspace permissions, the GoodData Python SDK does provide specific

full_load

vs.

incremental_load

workflows [GitHub]. However, this applies to provisioning metadata and entities, not underlying dashboard analytical data)._ 👍 0 button 👎 0 button ℹ️ Tip: Mention @kapa.ai in the thread for follow-ups. 👀 Our Support team monitors all messages and will jump in if needed.

Hi Shelia, There are not any data loads necessary in GoodData Cloud. However, This is something that is used in GoodData Legacy. Are you referring to the legacy/platform product?

There are no data loading processes in GoodData Cloud. You connect your databases and then just query them directly. You can find information on this here: gooddata.ai/docs/cloud/connect-data