To GoodData homepage

Replies posted by Robert Moucha

29 Replies

kentteo
kentteoNew Participant
asked in SDKs and APIs

Getting error during Init copy-extra-drivers

The contents of ContainerConfig makes no difference. It simply shows the configuration of the container used during the image build. The older image was created on regular docker environment, while the newer one was created with Buildkit feature turned on. You could achieve the same result when building the image the following way: DOCKER_BUILDKIT=0 docker build -t myimage .The important thing is how the structure of the drivers looks like within the image itself. Every driver needs to be stored in extra subdirectory of /data , for example (the contents may differ depending on the drivers you have):/data├── BIGQUERY│   ├── animal-sniffer-annotations-1.20.jar│   ├── annotations-4.1.1.4.jar...truncated...│   ├── third-party-licenses.txt│   └── threetenbp-1.5.2.jar├── DREMIO│   ├── dremio-jdbc-driver-3.0.6-201812082352540436-1f684f9.jar│   ├── dremio-snowflake-plugin-20.1.0.jar│   ├── dremio-snowflake-plugin.jar│   └── dremio-verticaarp-plugin.jar├── DRILL│   └── drill-jdbc-all-1.18.0.jar

6
R
GoodData Employee
1 year ago
kentteo
kentteoNew Participant
asked in SDKs and APIs

Getting error during Init copy-extra-drivers

My wild guess would be that the image was built on another architecture than you are running kubernetes cluster.Is there a chance that the image was made on amd64 arch while running on arm64 or vice versa?Please check using:docker image inspect <MASKED>/gooddata-cn-extra-drivers@sha256:ff8b4ef16e4242dda027f3c0c4cda937ab58a36bd10abd11498e7b9ca4473bfcand look for "Architecture" attribute. This value must match nodeInfo.architecture of your Kubernetes nodes.There are ways how to build multi-arch image, but the build process is more complex. EDIT: Currently we support only amd64 builds because Apache Pulsar doesn’t offer arm64 images yet. So your kubernetes cluster must also run on amd64 CPU architecture and custom driver images must be built for amd64 arch as well.

6
R
GoodData Employee
1 year ago
Z
zmazulaKnown Participant
asked in Administration

Performance tuning

Hi Zoltán,there are multiple hints how to increase performance:grant more CPU/Memory resources to services running in k8s. There are *.resources.limits set per component (asterisk is a placeholder for many various components, see values.yaml in our helm chart). The default limits are set rather low to make sure it will also run on smaller HW). If you have issues with slow API requests (those xhr-type requests in browser), focus on the following components: afmExecApi, metadataApi and resultCache. Network latency - if your cluster runs in remote region, network latencies can be bigger. You may want to deploy your cluster closer to your clients. Monitoring - watch closely performance metrics and identify possible bottlenecks. Authentication - we are aware of some inefficiencies in authentication processing. These issues were fixed after the version 1.7.2 was released and will be included in the next version. It should lower the TTFB metric.Regards,Robert Moucha

10
Z
1 year ago
d.novikov
d.novikovParticipating Frequently
asked in GoodData.UI

How to configure CORS for the /api/profile path?

Hello, a brief update:CORS setup works fine as far as you do NOT add any cors-related annotations to your ingress annotations (simply remove `nginx.ingress.kubernetes.io/enable-cors`,  `nginx.ingress.kubernetes.io/cors-allow-headers` and `nginx.ingress.kubernetes.io/cors-allow-origin` from your customized values file and update your deployment. Removing these annotations is necessary to keep responses unmodified because now the CORS is completely managed by backend services.Then, set `allowedOrigins` to list of allowed origin URLs; don’t forget to include schema and also port if it differs from default value for given schema (80 for http and 443 for https).These two settings should be sufficient to make the CORS working. Please us know if it helped.

13
R
GoodData Employee
1 year ago
d.novikov
d.novikovParticipating Frequently
asked in GoodData.UI

How to configure CORS for the /api/profile path?

It seems there’s unexpected interference between cors setup on ingress level and cors setup on organization level, so it really doesn’t work as expected. We will fix in in one of future releases, but till then there is the following workaround:Add the following annotations to helm chart values and redeploy gooddata-cn:ingress: annotations: nginx.ingress.kubernetes.io/enable-cors: "true" nginx.ingress.kubernetes.io/cors-allow-headers: "X-GDC-JS-SDK-COMP, X-GDC-JS-SDK-COMP-PROPS, X-GDC-JS-PACKAGE, X-GDC-JS-PACKAGE-VERSION, X-GDC-VALIDATE-RELATIONS, DNT, X-CustomHeader, Keep-Alive, User-Agent, X-Requested-With, If-Modified-Since, Cache-Control, Content-Type, Authorization" nginx.ingress.kubernetes.io/cors-allow-origin: "https://localhost:8443, https://another.domain.com"The “cors-allow-origin” annotation should contain comma-separated list of all domains where the CORS should be enabled for all organizations you have. Note, that you can’t control which domain should be enabled f

13
R
GoodData Employee
1 year ago
d.novikov
d.novikovParticipating Frequently
asked in Data models

Trying to load a big database into a model leads to browser tab crash

It’s not a matter of db table size - the more important is an overall complexity of your data model - number of tables and views, number of table columns, number of table relations (foreign keys).You may filter tables on common prefix to load a subset of tables that are subject of your interest. Later on, you may append another portion of your model.We are aware of performance issues in modeler, UI starts to slow down when number of datasets exceeds two or three hundreds. Some improvements were done in recent release but there are still some cases that need to be improved.

4
R
GoodData Employee
2 years ago
I
IVèSParticipating Frequently
asked in Administration

Fail to install GoodData in a Self-Hosted Kubernetes cluster

Hi, it’s a great progress. Just your way of nginx validation may not be sufficient. I recommend deploying some simple service (like https://github.com/stefanprodan/podinfo) and create a real ingress pointing to this deployment:# deploy test app (or use another way of deployment, see the GitHub page)kubectl apply -k github.com/stefanprodan/podinfo//kustomize# create ingress (update podinfo.example.com to some real hostname on 172.21.0.12)kubectl  create ingress podinfo --rule="podinfo.example.com/*=podinfo:http" --class=nginx# check the ingress works through LBcurl http://podinfo.example.com# (it should return a short JSON document) 

8
I
2 years ago
I
IVèSParticipating Frequently
asked in Administration

Fail to install GoodData in a Self-Hosted Kubernetes cluster

Hello, thank you for detailed steps explaining the installation process. It will make troubleshooting much easier.issue 1) Ingress controllerDefault setup for ingress-nginx assumes you have some external loadbalancer that can be managed by k8s. This works finein public cloud environments, but on bare metal or in private clouds it is not always possible. Refer to https://kubernetes.github.io/ingress-nginx/deploy/baremetal/ for details how to cope with this situation and how to properlyconfigure ingress-nginx controller in your environment.issue 2) unable to recognize "organization.yaml"Organization is a custom resource, whose definition (so-called CRD) is installed as a part of gooddata-cn helm chart release.So you must install gooddata-cn chart first, then create your organization.The same applies for your attempt to create user using `/api/auth/users` - this API is provided by gooddata-cn issue 3) timed out waiting for the conditionThis issue may have many reasons; the most common is

8
I
2 years ago
A
aleks2022New Participant
asked in Data sources

GD Cloud Native Docker - connectivity issue (timeout)

Hi, if I understand correctly, you’re behind some corporate proxy, so you can’t access remote resources directly?There’s no explicit support for proxied connections in GoodData.CN and I believe it will not work out of the box, even if you try set usual proxy-related variables. 

7
R
GoodData Employee
2 years ago
A
AnjaliNew Participant
posted in Administration

404 error on gooddata-cn kubenetes installation

I see, so the problem is that the metadata-api pods are not Ready, because the application doesn’t start. That’s the reason of missing endpoints, as they are created by Service when some pod matching the Service selector becomes Ready.Please check the container logs of these pods using (they are called like `gooddata-cn-metadata-api-xxxxxxxx-yyyyyyyy`) to see why they don’t start. It’s also possible that these pods are subject of resource starvation - could you please share how many nodes you have and how much RAM and CPU they have?

4
R
GoodData Employee
2 years ago
A
AnjaliNew Participant
posted in Administration

404 error on gooddata-cn kubenetes installation

Hi, assuming you’re using a new ingress-ngnix helm chart (>4.0.0) with older GoodData.CN helm chart (1.3.0). There’s documented step related to setting proper annotations related to ingress class. Please set this annotation both to .ingress.annotations and to .dex.ingress.annotations as shown in example. Note - another (better) solution would be to upgrade to gooddata-cn 1.4.0 where these annotations should not be set because resources are using modern `ingressClassName` attribute supported by networking.k8s.io/v1 Ingress. 

4
R
GoodData Employee
2 years ago
A
AnjaliNew Participant
posted in About GoodData

GoodData.CN on ARM based VM's

Hi, we distribute docker images only for amd64 architecture. Also, the Apache Pulsar images are available for am64 only, because incompatibility with one of its build depenencies.We plan to start building our multiarch images (containing amd64 and aarch64), but it doesn’t resolve the issue with Pulsar anyway.Therefore you’d need both ARM and Intel-based nodes anyway, which is not much cost-effective solution. 

2
A
2 years ago
ashok
ashokKnown Participant
asked in Data models

Ubable to import models form GDCloud to GDCN

Models built on GoodData Cloud are not compatible to GoodData.CN models. Therefore it is not possible to import GoodData Cloud models to GoodData.CN. 

1
R
GoodData Employee
2 years ago
ashok
ashokKnown Participant
asked in Administration

Unregistered redirect_uri using Built-in OIDC Identity Provider

Ah, my bad . I misplaced the configuration options I advised to a different section. These three (proxy-buffer-size, client-body-buffer-size, and use-forwarded-headers) do NOT belong to controller.service.annotations, but to controller.config.So the correct structure is:controller: config: # This resolves a possible issue with big headers proxy-buffer-size: '16k' # Improve performance of requests with a large body client-body-buffer-size: '1m' # use X-Forwarded-* received from ELB - important for proper propagation # of LoadBalancer host, port, and schema use-forwarded-headers: 'true' service: annotations: # some other annotations...I’m very sorry.

4
ashok
2 years ago
ashok
ashokKnown Participant
asked in Administration

Unregistered redirect_uri using Built-in OIDC Identity Provider

Hello @ashok, the redirect_url has wrong schema (http instead of https). This error is caused by the fact the Ingress controller doesn’t pass X-Forwarded-* headers from upstream load balancer to application.Please add the following annotations to ingress-nginx service helm chart (using values-ingress.yaml file) and perform rollout restart of ingress-nginx deployment: controller:  service:    annotations: # This resolves a possible issue with big headers proxy-buffer-size: '16k' # Improve performance of requests with a large body client-body-buffer-size: '1m' # use X-Forwarded-* received from ELB - important for proper propagation # of LoadBalancer host, port, and schema use-forwarded-headers: 'true' # some other annotations...Especially the “use-forwarded-headers” is very important.Reapply the ingress-nginx helm chart with these updated values file and restart ingress-nginx deployment using “kubectl rollout restart deployment ...” command.Then,

4
ashok
2 years ago
ashok
ashokKnown Participant
asked in Administration

How to properly Terminate SSL using AWS ALB with ACM

Please add the following annotations to respective places in customized-values-gooddata-cn.yaml used for deploying gooddata-cn helm chart:Keep in mind you need to “merge” these two settings to existing keys.dex: ingress: annotations: kubernetes.io/ingress.class: nginxingress: annotations: kubernetes.io/ingress.class: nginxThis change is necessary for a proper operation with the recent versions of ingress-nginx. We will update documentation for version 1.3 and address this issue in the upcoming release. 

4
ashok
2 years ago
D
dasteinParticipating Frequently
asked in Dashboards and Reports

Can't create Insights, results cache crashing

Hi,its caused by too many cells generated during data transformation of your result (882500 cells) and this value won’t fit to memory of the component result-cache. We plan to limit the results at earlier stage so the OOM should not happen and error will be reported sooner. The root cause is that your report probably contains too many distinct values in some of your dimensions. Now, you may try increasing memory of result-cache by setting this tunables in helm chart (see the defaults we have):# Defaults from values.yaml, update according to your needsresultCache: resources: limits: memory: 700Mi requests: memory: 400Mi The solution above will not help in all cases, larger result sets may fail, so I suggest to rework your metric to return less distinct values.Regards,Robert

2
R
GoodData Employee
2 years ago
D
dasteinParticipating Frequently
asked in Data sources

Can't initialize BigQuery connection - "Headless Exception" error

FYI, in the upcoming release 1.4, this extra step with file removal will not be required.

18
J
2 years ago
D
dasteinParticipating Frequently
asked in Data sources

Can't initialize BigQuery connection - "Headless Exception" error

@jku - thank you for the full exception stack, it was actually very helpful. The error shows a classloading confilict in gRPC library. Are you sure you properly followed a procedure described in https://www.gooddata.com/developers/cloud-native/doc/1.3/administration/data-sources/supported/bigquery/#prerequisite and removed the gRPC API library from Simba BigQuery driver, before building an image with db-drivers?cd <folder where driver is unpacked>rm grpc-api-*.jar  

18
J
2 years ago
D
dasteinParticipating Frequently
asked in Data sources

Can't initialize BigQuery connection - "Headless Exception" error

For us, it works fine with the older version 1.2.4.1007 as well as with the latest 1.2.19.1023.You’re using 1.2.16.1020 so I guess it should work as well. 

18
J
2 years ago
D
dasteinParticipating Frequently
asked in Data sources

Can't initialize BigQuery connection - "Headless Exception" error

Indeed, there’s an error in /api/actions/dataSource/test API preventing a proper operation of this test API on BigQuery datasources.It also explains why the tempfile is empty, because token is mistakenly ignored. I have filed internal issue and we will fix it in upcomming release. But it still doesn’t explain the original error java.awt.HeadlessException induced by bigquery driver’s attempt to render user login dialogue. Could you please try to reproduce the issue the following way:delete your data source (DELETE /api/entities/dataSources/YOUR-BQ-DATASOURCE-ID) construct body of createDatasource API request:{ "data": { "attributes": { "name": "YOUR BQ DATASOURCE", "url": "jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;ProjectId=sandbox-analytics-01;OAuthType=0", "schema": "dbt_production", "type": "BIGQUERY", "username": "", "password": "", "token": "ewog..." }, "id": "YOUR-BQ-DATASOURCE-ID", "type": "dataSource" }}Submit thi

18
J
2 years ago
ashok
ashokKnown Participant
asked in Administration

How to properly Terminate SSL using AWS ALB with ACM

Hello Ashok,the Nginx Ingress Controller works well with classical AWS ELB (Elastic load balancer). AWS ALB (Application load balancer) should be controlled by https://github.com/kubernetes-sigs/aws-load-balancer-controllerWhile it is possible to have ALB in front of NGINX ingress controller deployment (see this issue or more detailed blog post), we recommend to use ELB instead, because it is far more easy to configure.Using ALB directly is not supported as it lacks some functionality (notably regex processing of paths)

4
ashok
2 years ago
A
AnjaliNew Participant
asked in Administration

SSO configuration on GoodData.CN using SAML protocol

SAML protocol is not supported, GoodData.CN supports OAuth 2.0 Authorization code grant flow only, via OpenID Connect (OIDC).See https://www.gooddata.com/developers/cloud-native/doc/latest/administration/auth/external-oidc/ for details. 

1
R
GoodData Employee
2 years ago
N
NickKKnown Participant
asked in Administration

GoodData created 21k threads!

For the record, the “graceful” way how to restart supervised service is:s6-svc -r /run/s6/services/SERVICE-NAMEwhere SERVICE_NAME is one of supervised services:afm-exec-api aqe auth-service cache-garbage-collector dex metadata-api nginx postgres pulsar redis result-cache scan-model sql-executorBut in the most cases, simple restart of the whole gooddata-cn-ce container is much easier and recommended solution.

9
J
GoodData Employee
2 years ago
N
NickKKnown Participant
asked in Administration

Can I use my own domain name with GoodData.CN (evaluation)

Changing the organization hostname will be possible in Kubernetes deployment of GoodData.CN by modifying the “Organization” custom resource responsible for the whole organization lifecycle.In standalone Docker deployment of GoodData.CN Community Edition this change is not possible. There’s a single hostname (“localhost”) that serves as an endpoint for the pre-created organization. This limitation can be worked around by putting the docker container behind a well-configured reverse proxy, as you have already shown in some of your previous comments. This configuration allows you to expose the container to your teammates so you can collaborate on this shared environment. Some future versions of GoodData.CN CE would natively support a custom hostname, but this is not our priority at this moment because we are laser-focused on the upcoming Kubernetes deployment release.Regards,Robert

14
R
GoodData Employee
2 years ago
Welcome to the GoodData Community!Cookie settings