Release Notes - GoodData.CN

GoodData.CN 2.0.0

Related products: GoodData.CN
GoodData.CN 2.0.0

Released July 14th, 2022.

New Features

  • We added a new Developer settings page that lets you perform the following actions within the GoodData web UI:
  • Secondary label functionality has been expanded upon. You can now add multiple labels to an attribute and use these labels to customize the way your data is presented in visualizations. See Manage Labels.
  • You can now sort attributes in ascending or descending order using one of its labels, giving you an additional layer of control over the way data is displayed in your visualizations. See Sort Labels.
  • We added support for M:N mapping. See Many-to-Many in Logical Data Models.
  • We have reworked the data source administration UI in the Data modeller. Connecting to and editing connections to data sources should be more user-friendly now.
  • Time filtering in visualizations has been expanded. You can create multiple filters with different dimensions for your visualizations, exclude current periods to remove incomplete data, and compare your data with previous time periods. See Filter Visualizations by Attributes or Dates.
  • Pivot tables now support subtotals. See Table Totals.
  • Metric editor now supports multiple tabs even when it is opened in the Analyze tab.
  • Multiple axis titles are now displayed on charts that have multiple Y-axis (there are multiple items in the View By bucket).
  • You can now customize the URL where the issued OAuth2 Authorization Code is sent from the OIDC provider to the GoodData backend (the so-called "OIDC redirect URL").
  • Expanded support for Snowflake:
    • We now support week granularity of the date dimension.
    • We now support lower-cased DB and WAREHOUSE names.
    • We now support proxy settings for JDBC URL for Snowflake data sources. See Permitted parameters.

Fixes and Updates

  • Apache Pulsar has been updated to version 2.9.2.
  • When you stack a bar chart by an attribute, the bars are now be correctly sorted by their total value.
  • You are now be able to embed a dashboard with filtered tags than contain spaces between words.
  • Fixed a bug where the collectLabelElements API endpoint would return elements even if a given offset was greater than total count of elements.
  • The Hyperlink in attribute label option in the Drill to URL are now be available in Dashboards as well.

Breaking Changes

  • PostrgreSQL database in GoodData.CN Community Edition all-in-one image was upgraded to version 13. If you're upgrading from Community Edition 1.7 to 2.0, and want to migrate all your data to the new version. See Upgrade Guides.
  • The API is now versioned. All /api/<endpoint> calls have to be updated to /api/v1/<endpoint>. See Versioning.
  • The pivot table API layout has been reworked as a consequence of the addition of subtotals to pivot tables. The grandTotals and includedDimensions objects have been renamed to totals and totalDimensions, among other breaking changes.
  • The primary property in logicalModel declarative API has been deprecated. Attributes are now mapped directly to database columns using the sourceColumn property in Logical Data Model declarative and entity APIs.

Deprecation Announcements

  • We deprecated support for Kubernetes older than version 1.21.
  • We deprecated the tablePrefix and viewPrefix parameters in the API endpoint generateLogicalModel.
  • We deprecated the API endpoint license.

Get the Community Edition

Pull the GoodData.CN Community Edition to get started with the latest release:

docker pull gooddata/gooddata-cn-ce:2.0.0

Upgrade Guides

Follow these guides if you want to upgrade your GoodData deployment from 1.7 to 2.0 without losing your data in the process.

For GoodData.CN (Kubernetes Edition) users, follow the instructions in Upgrade GoodData.CN to 2.0.

For GoodData.CN Community Edition users:

Upgrade GoodData.CN to 2.0

Before upgrading gooddata-cn helm chart, upgrade your Kubernetes cluster to version 1.21 (up to 1.24).

Upgrade Apache Pulsar deployment to version 2.9.2.

Preload updated Custom Resource Definition (CRD) for Organization resources. Due to the helm command limitation, it is impossible to update CRD automatically by Helm. Therefore it is necessary to update this CRD manually by following this procedure:

  1. Download and extract gooddata-cn helm chart to an empty directory on local disk.

    helm pull gooddata/gooddata-cn --version 2.0.0 --untar
  2. Update modified CRD in the cluster where the older GoodData.CN release is deployed:

    kubectl apply -f gooddata-cn/crds/organization-crd.yaml
  3. You can clean local extracted helm chart

    rm -r gooddata-cn

Upgrade Postgresql-ha Helm Chart

If you're using external PostgreSQL database (when you deployed gooddata-cn helm chart with option deployPostgresHA: false), you can skip this step because it is not relevant for your deployment.

If you're using embedded PostgreSQL database (with deployPostgresHA: true set in values.yaml), you will need to perform the following process to upgrade postgresql-ha helm chart. Due to the nature of the upgrade, this action will cause a service disruption. Please schedule maintenance window for this operation.

  1. Scale down PostgreSQL StatefulSet to single replica:

    kubectl -n gooddata-cn scale --replicas=1 statefulset gooddata-cn-db-postgresql
  2. Perform GoodData.CN helm chart upgrade to version 2.0.0 according to the general upgrade guide, but append the following extra parameters on command line of helm upgrade command:

    --set postgresql-ha.postgresql.replicaCount=1 \
    --set postgresql-ha.postgresql.upgradeRepmgrExtension=true
  3. Reapply the GoodData.CN helm chart version 2.0.0 using helm upgrade command, but without two extra paramteters mentioned in the previous step. It will automatically scales up PostgreSQL StatefulSet to three pods.

Upgrade GoodData.CN Community Edition to 2.0 - Including Your Organization Layout

As an administrator, you may want to import an organization dump from the version 1.7 / 1.7.x deployment into the version 2.0 deployment. Before you can do so, you need to add sourceColumn to an attribute definition, that is the same as for label with flag primary=true, then remove the label with flag primary=true and finally remove flag primary=false from other labels. The label with flag primary=true is no longer part of the declarative document, and it will be automatically recreated on import from the attribute definition.

PostgreSQL database data are deleted from the volume as part of procedure. In our instructions we are using gd-data:/data as an example.

Steps:

  1. If you have not done so already, create a /tmp/organization-layout.json JSON dump from your GoodData.CN 1.7.x organization layout. See Back Up the Organization.

  2. Use the following command to create an updated version of the organization-layout.json dump file:

    jq ".workspaces[].model.ldm.datasets[].attributes[] |= (
    .sourceColumn = (.labels[] | select (.primary == true) | .sourceColumn) |
    .labels = [ (.labels[] |
    if (.primary == false) then
    del(.primary)
    else
    empty
    end)
    ])" /tmp/organization-layout.json > /tmp/updated-organization-layout.json
  3. Stop your GoodData.CN 1.7 instance.

  4. Remove old PostgreSQL data and prepare volume for start of GoodData.CN 2.0:

    docker run --rm -v gd-data:/data busybox:1.34 rm -rf /data/postgresql
    docker run --rm -v gd-data:/data busybox:1.34 rm -rf /run/s6
  5. Start the GoodData.CN 2.0 instance.

  6. Import the updated-organization-layout.json file into your GoodData.CN 2.0 deployment. See Restore the Organization.

Upgrade GoodData.CN Community Edition to 2.0 - Including Your Organization Layout and Database

In GoodData Community Edition version 2.0 the PostgreSQL database has been updated to version 13. This upgrade guide lets you update the database without losing any of your data.

Please note that you can upgrade the PostgreSQL database only if your current GoodData Community Edition is using a named docker volume to keep the data. In our instructions we are using gd-data:/data as an example.

Steps:

  1. Export your organization layout using the API.

    This will let you verify the upgrade was successful at a later point in this procedure.

  2. Turn off your current Community Edition deployment:

    docker stop -t 60 <container-running-old-version>
    docker rm <container-running-old-version>
  3. Prepare environmental variables:

    PGDATABCK=/data/postgresql-bck
    PGDATAOLD=/data/postgresql-old
    PGDATANEW=/data/postgresql
    CURR_DATETIME="$(date +'%Y-%m-%d_%H:%M')"
    CHECK_DATA="/data/check_${CURR_DATETIME}"
    UPGRADE_DATA="/data/upgrade_${CURR_DATETIME}"
  4. Create a backup. This is for rollback purposes only and can be deleted after a successful migration:

    docker run --rm -v gd-volume:/data busybox:1.34 cp -a $PGDATANEW/ $PGDATABCK
  5. Create another version of the database which is going to be upgraded:

    docker run --rm -v gd-volume:/data busybox:1.34 mv $PGDATANEW/ $PGDATAOLD
  6. Ensure the upgrade is possible:

    docker run --rm -v gd-volume:/data \
    -e PGDATAOLD=$PGDATAOLD -e PGDATANEW=$PGDATANEW \
    --workdir="${CHECK_DATA}" \
    tianon/postgres-upgrade:11-to-13 --link --check \
    -o "-c config_file=/data/postgresql/postgresql.conf -c hba_file=/data/postgresql/pg_hba.conf"
  7. Upgrade:

    docker run --rm -v gd-volume:/data \
    -e PGDATAOLD=$PGDATAOLD -e PGDATANEW=$PGDATANEW \
    --workdir="${UPGRADE_DATA}" \
    tianon/postgres-upgrade:11-to-13 --link \
    -o "-c config_file=/data/postgresql/postgresql.conf -c hba_file=/data/postgresql/pg_hba.conf"

    # Set correct user and group for scripts created by upgrade procedure
    docker run --rm -v gd-volume:/data busybox:1.34 chown -R 102:103 "$UPGRADE_DATA"
  8. Launch GoodData Community Edition 2.0. Ensure it starts properly.

  9. Collect database statistics:

    docker exec --workdir "$UPGRADE_DATA" -u postgres <container-running-version-2.0> bash -c "./analyze_new_cluster.sh"
  10. You can verify the contents of the organization by creating an export of the organization layout and comparing it to the export you created in step 1.

  11. If the upgrade was successful, you may delete the backup:

    docker exec <container-running-version-2.0> rm -rf $PGDATABCK

    and the base data for the upgrade:

    docker exec --workdir "$UPGRADE_DATA" <container-running-version-2.0> bash -c "./delete_old_cluster.sh"

Rollback

In case something goes wrong, you can use the backup you have created in step 4 to restore your database.

Steps:

  1. Stop the current container running your GoodData Community Edition:

    docker stop -t 60 <container-running-old-version>
    docker rm <container-running-old-version>
  2. Restore your data:

    docker run --rm -v gd-volume:/data busybox:1.34 rm -rf $PGDATANEW
    docker run --rm -v gd-volume:/data busybox:1.34 mv $PGDATABCK/ $PGDATANEW
  3. Launch the previous version of your GoodData Community Edition.