GoodData.CN 3.0.0

Released October 19th, 2023.

New Features

Explicit Workspace Data Filters

We've introduced a new panel in the Logical Data Modeller to help you manage workspace data filters. Within the “view details” dialog, a new “Data filtering” tab now allows users to toggle workspace data filters for designated datasets and align them with columns in the source table/view. Instead of merely matching column names, the mapping of these filters necessitates explicit activation and dataset-specific mapping.

This enhancement marks the beginning of our vision to make workspace data filters manageable via the application's frontend, moving beyond exclusive API access.

Learn More:

Set Up Data Filters in Workspaces

Enhanced Metadata in Analytics Entities

We have added new metadata to analytics entities - analytical dashboards, visualization objects, dashboard plugins, and metrics. These entities will now include the following fields:

• createdAt: Displays the creation timestamp.

• createdBy: Shows the original creator.

• modifiedAt: Shows the latest edit timestamp.

• modifiedBy: Points to the last editor.

To ensure smooth integration, we're updating the metadata database in this release. It includes adding new fields to related tables. For existing entities, createdAt gets the local time, while other fields (createdBy, modifiedAt, modifiedBy) remain null.

Analytical Designer and Dashboards will now list insights by modification date, with the latest on top. Tabs above allow filtering by the current user's creations and all workspace insights. Note that the default tab shows only new insights due to metadata migration not populating createdBy.

Learn More:

Extended Metadata

Themes and Palettes for Individual Users

Theme and color palette functionality was extended to also enable use in userSettings, allowing you to create unique visual styles for individual users and user groups.

Learn More:

Create Custom Color Palettes
Create Custom Themes

Improved Sharing

We have improved dashboard sharing options. You can now effortlessly share a dashboard with all workspace users in a single click. In previous versions, creating user groups with all users was required, which could be cumbersome and challenging to manage. This feature enables granting various dashboard permissions (View, Edit, Share) just like when granting to individual users or user groups.

Learn More:

Share Dashboards

Transposing Pivot Tables

We're giving you more control over positioning attributes, headers, and metrics among columns and rows in pivot tables. You can now easily change the orientation of pivot tables (e.g. show the headers, contents, and totals of columns in rows and vice versa) or put metrics into rows using a simple drop-down selector in the table's configuration.

Learn More:

Pivot Tables

Column and Row Totals in Pivot Tables

We have expanded the functionality of table totals. You can now use aggregate functions, i.e. both totals (showing the total sum of the whole column or row) and sub-totals (showing aggregates within individual attributes), for both columns or rows. For example, you can generate a column that shows the total sum of values of individual rows.

Learn More:

Pivot Tables

Improved Security

We've enhanced GoodData's security features. Administrators in your organization can now limit users from creating user API tokens, ensuring that users must log in via single sign-on (SSO) for each session. In prior versions, users could generate their own API tokens, bypassing SSO after the initial login.

Change in Default:

By default, only administrators (users with the organization.MANAGE permission) can now create new API access tokens. Existing tokens remain valid.

Learn More:

Create an API Access Token

Introducing FlexQuery

We're excited to announce FlexQuery, our newly introduced service that enhances the transformation and caching of data delivered by the SQL Executor Service. Designed with engineers in mind, FlexQuery aims to significantly reduce query times and save on costs by minimizing expensive cloud data warehouse query processing.

Please be aware that updating to GoodData.CN 3.0 requires you to follow some specific steps related to FlexQuery.

Learn More:

Our Vision for the Next Generation of BI and Analytics

Other Changes

• We have improved the SQL generated for date range conditions to boost performance.

Fixed Bugs

• IDs that contain the + character should no longer cause issues when being referenced in the url of an API call.

• We resolved an issue where certain workspace hierarchy permission configurations could result in you being unable to open the metric editor in a child workspace.

• We have resolved issues with the metric editor in children workspaces.

Depreciations

Physical Data Model

We've streamlined our modeling processes. Previously, a separate physical data model (PDM) was built atop the data source, while logical data models (LDM) were set for individual workspaces. Now, LDMs will encompass PDM details, centralizing data within a single workspace and simplifying SQL-based dataset interactions. While backend changes impact the REST API's interaction with LDM and PDM, frontend user interactions remain unchanged. Users should transition LDM backups to the new format and cease using the deprecated PDM and its API endpoints.

If you have saved declarative workspace definitions in a location separate from the standard storage, which can be used for backups, we advise creating a fresh backup, particularly of the LDM workspace definitions, after a successful upgrade to GoodData.CN 3.0.0. This new backup will include the updated LDM.

If you have a workspace declarative model stored externally, i.e. not present in the deployment, upload this model along with the associated data source and its PDM. Then, proceed to download the updated version of the workspace declarative model.

Community Edition

The GoodData.CN Community Edition has been discontinued for GoodData.CN versions 3.0 and above. GoodData Community Edition versions prior to 3.0 will continue to function without change.

If you're interested in exploring our analytical platform, you can register for a free trial at https://www.gooddata.com/trial/.

If you have any questions, please feel free to contact us via https://www.gooddata.com/contact/.

Developer Tools

Python SDK

Please update Python SDK to the latest version. Using an older version of Python SDK together with GoodData.CN version 3.0 or higher could cause issues with the Logical Data Model.

GoodData.UI

The following updates to GoodData.UI have been released:

• 9.0.1
  This hotfix introduces support for JWT authentication.

• 9.1.0

  • Added support for pivot tables, allowing you to transpose column headers to rows for more flexible data analysis.

  • Pivot tables allow metrics to be placed in rows instead of columns.

  • The attribute filter is optimized to fetch element counts only for child filters, improving performance. Because it does not fetch the total count of elements by default, its overhead is reduced.

  • Sdk-backend-tiger now supports the following entity properties: createdAt, createdBy, modifiedAt, and modifedBy for the following analytical objects: insight, analytical dashboard, dashboard plugin, and metric.

  • Sdk-backend-tiger now supports JWT authentication.

  • Added support for Waterfall chart visualizations.

  • The dashboard header element had its z-index changed from 100 to 6000. For reference the z-indicies of the dashboard overlay elements are as follows:

    • Overlays in Dashboard component  = 5000

    • Overlays in Dashboard header component = 6000

    • Overlays in Dashboard component in conflict with filter bar = 6000

Upgrade Guide

FlexQuery - Caching and Computation Engine Upgrade

Starting from version 3.0, the default caching and computation engine of GoodData.CN has undergone significant changes, introducing a new set of services in the deployment. These changes include:

1. quiver-xtab: This service is responsible for processing computations.

2. quiver-cache: Manages the storage of both raw and result cache data (data extracted from data sources and subsequently transformed into reports) as well as label elements (data for filters). Until version 3.0, all the caches were stored in redis.

3. etcd: This database supports synchronization between the quiver-* services.

Note that quiver is an internal name for FlexQuery.

As raw, result, and label element data caches are no longer stored in redis (only their metadata still are), we recommend reviewing your redis memory utilization and adjusting your memory resources as needed. If you prefer to continue using the old caching flow, you can use the Helm chart options deployQuiver: false and useInternalQuiverEtcd: false. However, note that this option will be deprecated in the next release.

Managing Caches

The new caching system, by default, stores caches in the quiver-cache service only. When the service's file storage is fully utilized (set to 1GB by default), it triggers the eviction of the least recently used caches to maintain file storage within its limit. The service offers several configuration options for managing caches:

• quiver.storage.cache.diskSize and quiver.storage.cache.diskCacheSize allow you to adjust the maximum internal storage size in the service.

• quiver.limitFlightCount defines the maximum number of caches that can be held in the system. Note that increasing this value may decrease performance.

• durableStorageType define durable caches and ensures their durability even after quiver-* services restart. Note that every change of durableStorageType requires wiping etcd, which holds storage configuration from the last time. You can achieve this by running etcdctl del --prefix <deployment_namespace_name> on one of the etcd PODs.
  Use one of the following values:

  • "" (no durable storage): All caches are held by the quiver-cache POD.

  • "S3": Durable caches are stored in the defined AWS S3 bucket (see below).

  • "FS": Durable caches are stored on storage defined by storageClassName, which must support the ReadWriteMany access mode. 

• s3DurableStorage.s3Bucket: The AWS bucket name for storing caches.

• s3DurableStorage.s3BucketPrefix: Custom bucket prefix.

• s3DurableStorage.s3Region: The AWS region (the default is "us-east-1").

• s3DurableStorage.s3AccessKey: AWS access key ID of the IAM account with access to the S3 bucket.

• s3DurableStorage.s3SecretKey: AWS secret access key of the IAM account with access to the S3 bucket.

• s3DurableStorage.authType: Authentication type ("aws_tokens" | "aws_default" | "none," default is "aws_default"), respecting AWS credentials provider chain.

We recommend implementing monitoring, alerting, and regular defragmentation of etcd.

For a comprehensive list of configuration options related to the new caching system, see the Helm Charts Options section of our documentation.

Check That the Physical Data Model Removal Was Successful

With the deprecation of the physical data model, updating to GoodData.CN 3.0 now triggers an automatic enhancement of your logical data models (LDMs). This enhancement adds new data types, paths to tables, and references to workspace data filters. Our aim is to ensure that your LDMs operate seamlessly even after the physical data models are phased out from GoodData.

What should you do?
The update is automatic, but we recommend you check the logs after starting your metadata-api service. Look for the following lines:

• action=polling detail=ENABLE_PDM_REMOVAL_DEPRECATION_PHASE stateChange=offToOn state=start

• action=polling detail=ENABLE_PDM_REMOVAL_DEPRECATION_PHASE stateChange=offToOn state=success

These indicate the update was successful. Ensure the following line is NOT present in your logs:

action=polling detail=ENABLE_PDM_REMOVAL_DEPRECATION_PHASE stateChange=offToOn state=error

If this error occurs, delete your metadata-api pod, and after its automatic start check the logs again for this error. If the error persists, please reach out to GoodData support for help.

Please note that this update is a one-time process and will not recur on future pod restarts once successfully completed.

Upgrade GoodData.CN to 3.0

To upgrade Helm chart, follow the general upgrade guide.