Hi. I have created an EKS cluster and installed Gooddata-cn in June for testing purposes. Everything worked, and we could log in, create workspaces, connect to data sources, and run the demos. Well we have not used it for a while, and we noticed last week or so that we could not log in any more. The URL to the website leads to https://URL/appLogin?redirectTo=%2F, which displays. { title: "Internal Server Error", status: 500, detail: "Server-side problem. Contact support.", traceId: "b1fb49546a89f667" } When I check on pod status, I saw $ kubectl get pod -n gooddata-cn NAME READY STATUS RESTARTS AGE gooddata-cn-afm-exec-api-5c8cfc44ff-frr5c 1/1 Running 0 24d gooddata-cn-afm-exec-api-5c8cfc44ff-xpp94 1/1 Running 0 24d gooddata-cn-analytical-designer-7b6f6b8bbc-m4bxd 1/1 Running 0 24d gooddata-cn-analytical-designer-7b6f6b8bbc-xrx7q 1/1 Running 0 24d gooddata-cn-api-gateway-5c5985cd78-lw7nc 1/1 Running 0 22d gooddata-cn-api-gateway-5c5985cd78-lzfm9 1/1 Running 0 22d gooddata-cn-apidocs-b64c9d77-hjzjn 1/1 Running 0 24d gooddata-cn-apidocs-b64c9d77-wkfkk 1/1 Running 0 24d gooddata-cn-auth-service-f754c9d58-2zp2g 1/1 Running 0 24d gooddata-cn-auth-service-f754c9d58-56r2g 1/1 Running 0 24d gooddata-cn-cache-gc-28801020-zfjcn 0/1 Completed 0 148m gooddata-cn-cache-gc-28801080-v4xsf 0/1 Completed 0 88m gooddata-cn-cache-gc-28801140-5l6zs 0/1 Completed 0 28m gooddata-cn-calcique-755b6684fd-7sw4k 1/1 Running 0 24d gooddata-cn-calcique-755b6684fd-v7dn9 1/1 Running 0 24d gooddata-cn-dashboards-65bcc567f9-6xnxp 1/1 Running 0 24d gooddata-cn-dashboards-65bcc567f9-xhjnd 1/1 Running 0 24d gooddata-cn-dex-66d69794cd-75zpx 0/1 CrashLoopBackOff 6 (83s ago) 8m36s gooddata-cn-dex-66d69794cd-zrxsk 0/1 CrashLoopBackOff 6 (101s ago) 8m36s gooddata-cn-etcd-0 1/1 Running 0 24d gooddata-cn-etcd-1 1/1 Running 0 24d gooddata-cn-etcd-2 1/1 Running 0 24d gooddata-cn-export-controller-66778fccf-7s6wd 1/1 Running 0 24d gooddata-cn-export-controller-66778fccf-b7967 1/1 Running 0 24d gooddata-cn-home-ui-6c8c6b49db-6swxk 1/1 Running 0 24d gooddata-cn-home-ui-6c8c6b49db-nhqrq 1/1 Running 0 24d gooddata-cn-ldm-modeler-5565b5f669-qcp58 1/1 Running 0 24d gooddata-cn-ldm-modeler-5565b5f669-sm2kz 1/1 Running 0 24d gooddata-cn-measure-editor-745494c65-mnnr7 1/1 Running 0 24d gooddata-cn-measure-editor-745494c65-q6nk8 1/1 Running 0 24d gooddata-cn-metadata-api-69d7959c6c-9f5zf 1/1 Running 7 (16d ago) 22d gooddata-cn-metadata-api-69d7959c6c-d95z5 1/1 Running 7 (16d ago) 24d gooddata-cn-organization-controller-576b699cf6-lgwg4 1/1 Running 0 24d gooddata-cn-organization-controller-576b699cf6-x74ld 1/1 Running 0 24d gooddata-cn-pdf-stapler-service-6d84847969-kpsn2 1/1 Running 0 24d gooddata-cn-pdf-stapler-service-6d84847969-zt77f 1/1 Running 0 24d gooddata-cn-quiver-cache-79d6bd97c-5m9pv 1/1 Running 0 24d gooddata-cn-quiver-cache-79d6bd97c-m8ctg 1/1 Running 0 24d gooddata-cn-quiver-ml-7ddff85c7b-j4jrz 1/1 Running 0 24d gooddata-cn-quiver-ml-7ddff85c7b-rbzb9 1/1 Running 0 24d gooddata-cn-quiver-xtab-686bf87c5-86bln 1/1 Running 0 24d gooddata-cn-quiver-xtab-686bf87c5-gmhmt 1/1 Running 0 24d gooddata-cn-result-cache-6b54748767-fwxvh 1/1 Running 0 24d gooddata-cn-result-cache-6b54748767-w27w8 1/1 Running 0 24d gooddata-cn-scan-model-778c785f7b-mfd6j 1/1 Running 0 24d gooddata-cn-scan-model-778c785f7b-q44cs 1/1 Running 0 24d gooddata-cn-sql-executor-6fc88bd645-9tg4j 0/1 CrashLoopBackOff 7 (2m47s ago) 26m gooddata-cn-sql-executor-6fc88bd645-ms4b2 0/1 CrashLoopBackOff 7 (2m47s ago) 26m gooddata-cn-sql-executor-6fc88bd645-r2x8h 0/1 CrashLoopBackOff 7 (2m54s ago) 26m gooddata-cn-tabular-exporter-667845bfb-cqnqt 1/1 Running 0 24d gooddata-cn-tabular-exporter-667845bfb-m4vr9 1/1 Running 0 24d gooddata-cn-tools-69b445fd4c-wxcbs 1/1 Running 0 24d gooddata-cn-visual-exporter-service-6f55974db-8lf6r 3/3 Running 0 24d gooddata-cn-visual-exporter-service-6f55974db-q4z8j 3/3 Running 0 24d gooddata-cn-web-components-b65ff7dcc-2kvjh 1/1 Running 0 24d gooddata-cn-web-components-b65ff7dcc-ctjgw 1/1 Running 0 24d Dex and sql-executor are not ready and have been restarted more than 4000 times. (The above is after redeployment of the two today, but they have already been restarted multiple times). I looked into the logs but I have not pinpointed the issue yet. Could you please take a look at the logs and see what I need to do? Thanks in advance.

Hi Dongfeng, I reached out to you via DM

Hi, first of all, I have resolved the issue, so everything is good now. But I do have a couple of questions. I noticed that I had been using reader instance of the RDS cluster, instead of the writer instance. I changed it to writer instance and used "helm upgrade" to apply it. This made DEX working, but not SQL-Executor. The logs of SQL-Executor seemed to indicate issues with Pulsar. The I used "kubectl rollout restart" on Pulsar's bookie, broker, recovery and zookeeper. After this, I scaled down and back up the SQL-Executor, and it is working now. 1. I don't think I have changed RDS connections before. But is there any chance that GDCN works with RDS' reader instance? 2. What is the possible sequence that caused DEX and SQL-Executor to fail? 3. How is Pulsar get involved here? What is its role in SQL-Executor? Thanks

Hello @Dongfeng Lu, good job with resolving the issue! Let me answer your questions: 1. goodgata-cn uses only one endpoint for both reading and writing. 2. Dex was failing because it used the reader instance, same for SQL-Executor 3. Pulsar is used as message broker in gooddata-cn, most components uses it for inter-service communication. The components and use of pulsar are explained in our documentation https://www.gooddata.com/docs/cloud-native/3.19/install/architecture/microservices/#sql-executor, https://www.gooddata.com/docs/cloud-native/3.19/install/architecture/. Please let me know if you have any further questions.

Thanks, Martin, for the response.