https://www.gooddata.com/ logo
Join Slack
Powered by
We had to redeploy pulsar due to some cluster main...
# gooddata-cn
p
We had to redeploy pulsar due to some cluster maintenance and it seems this has caused our metadata-api pods (and other services that use Pulsar) to fail with the error: Caused by: org.apache.pulsar.client.admin.PulsarAdminException$NotFoundException: Namespace not found However, "kubectl get namespace -A" shows pulsar as one of the namespaces. Attaching the logs from my gooddata-cn-metadata-api pods. Any advice appreciated as to what might cause this error or how to resolve. Thanks so much 🙂
gooddata-cn-metadata-api-7666b6cd56-x8wz9_metadata-api.log
r
Hi Pete. The 
Namespace not found
error actually means Pulsar namespace, not Kubernetes namespace. The Apache Pulsar supports multi-tenant configurations with the following hierarchy: 
tenant / namespace / topic
The gooddata-cn automatically creates 
tenant
and 
namespace
at the installation so applications can register their topics within 
tentant/namespace
. During the cluster maintenance, you probably lost the pulsar configuration data. It's possible to recreate tenant and namespace manually
1. Connect to one of pulsar broker pods, using:
Copy code
kubectl -n pulsar exec -it -c pulsar-broker pulsar-broker-0 -- bash
2. create pulsar tenant with the same name as Kubernetes namespace where your gooddata-cn is installed. For example, if you installed gooddata-cn helm chart into 
gooddata-cn
k8s namespace, use the following command in broker pod to create pulsar tenant:
Copy code
bin/pulsar-admin tenants create gooddata-cn
You can check the tenant is present by running command 
bin/pulsar-admin tenants get gooddata-cn
, the output should look like:
Copy code
{
  "adminRoles": [],
  "allowedClusters": [
    "pulsar"
  ]
}
3. Create pulsar namespace with the name of the helm release name of gooddata-cn helm chart. Usually, the helm release name is the same as chart name (i.e. gooddata-cn), but you might use a different name during the chart installation.
Copy code
bin/pulsar-admin namespaces create gooddata-cn/gooddata-cn
The 
gooddata-cn/gooddata-cn
actually means 
<tenant-name>/<chart-release>
where 
<tenant-name>
is the tenant name from the step above and 
<chart-release>
is described earlier. After these steps, apps should register their topics and start correctly. You can check the progress by 
bin/pulsar-admin topics list gooddata-cn/gooddata-cn
. Fully working gooddata-cn should have the following topics registred:
Copy code
"<persistent://gooddata-cn/gooddata-cn/compute.calcique>"
"<persistent://gooddata-cn/gooddata-cn/metadata.cache-command>"
"<persistent://gooddata-cn/gooddata-cn/__change_events>"
"<persistent://gooddata-cn/gooddata-cn/sql.select.DLQ>"
"<persistent://gooddata-cn/gooddata-cn/sql.select>"
"<persistent://gooddata-cn/gooddata-cn/compute.calcique.DLQ>"
"<persistent://gooddata-cn/gooddata-cn/export-tabular.request>"
"<persistent://gooddata-cn/gooddata-cn/export-visual.request>"
"<persistent://gooddata-cn/gooddata-cn/result.xtab>"
"<persistent://gooddata-cn/gooddata-cn/caches.garbage-collect>"
"<persistent://gooddata-cn/gooddata-cn/data-source.change>"
"<persistent://gooddata-cn/gooddata-cn/metadata.model>"
"<persistent://gooddata-cn/gooddata-cn/result.xtab.DLQ>"
(export-visual and export-tabular might be missing on your deployment depending on gooddata-cn version you're using)
p
Thanks @Robert Moucha 😀
Open in Slack
PreviousNext
Powered by