https://www.gooddata.com/ logo
Join Slack
Powered by
hi crew, all day today I've been watching the spin...
# gooddata-cloud
t
hi crew, all day today I've been watching the spinner trying to load my model data to work on the LDM. Is there an outage or something? It's like this a LOT of the time as well, not like this is an isolated incident. This is on https://learnamp.cloud.gooddata.com/modeler/#/df566d0354e04365b659c67b12c34766 attempting to refresh data source so I can actually work on the mappings. 100% of the time results in: Connection failed An unexpected error has occurred
j
Hi Thomas, We are not aware of any issue on our end that could cause what you’re describing. Nor any other customer reached out with anything similar.
Do I understand correctly that currently, you’re not even able to refresh the data source? Are you able to connect to your DS without using GoodData?
t
correct, and yes
data source is aws so you know, pretty rock solid
is our LDM unusually complicated? We have near constant performance issues when trying to refresh the data source tbh
(or, apparent performance issues - long time on the spinner, often failing to load one way or another)
m
Hi Thomas, I went ahead and reviewed your model here: https://learnamp.cloud.gooddata.com/modeler/#/df566d0354e04365b659c67b12c34766 I placed the model into “Edit” mode and I saw that the examplecompanya datasource “spinning” and datasets took around 5 mins to load. However, the datasource loaded successfully and I was not able to replicate the Connection failed error on my side. Can you please try once again and let us know if you’re still running into any issues there? If you’re still experiencing the same issue - please can you tell us the steps to reproduce the issue and our team will be happy to investigate further.
t
thanks, I will try again this morning. I'd take the w at this point, I just need to actually get work done, but 5 mins to load is kind of unacceptable don't you think?
nope, same story. I follow the link you have posted, I click "edit", it shows the connection failed error
I have a feeling mine dies out before 5 mins, timing it now
omg it worked this time finally
can you guys provide any insight into why editing the ldm is borderline unusable for us?
surely, surely this is not business as usual for your product
j
Hi Thomas, I agree. Definitely not business usual. Pls let me investigate further and double-check internally. I was just trying to refresh data source in workspace 
df566d0354e04365b659c67b12c34766
and it took roughly 45 seconds for me.
But I’ll double-check what else might be in play here.
Is this by any chance user specific? Are all of your users observing the same behaviour? Also, as I wasn’t able to reproduce the issue in your environment, could you please record a HAR file while reproducing it and send it over to us? Feel free to use DM. This could greatly help us to investigate the issue.
t
good news, got repro
learnamp.cloud.gooddata.com.har
learnamp.cloud.gooddata.com.har
I'm sure it's not an all the time issue, but I will say that I haven't used the LDM editor since I raised this before, then first attempt to use it again and same story. So you can see why I have the impression that this is unusable
I've periodically tried throughout the day, and exactly the same story
j
Hi Thomas, unfortunately I cannot see anything much useful in the provided HAR file. However, I will pass the case to our L2 Technical Team for further verification. They can check the HAR file and explore possible options regarding what might be exactly happening here. In the meantime, could you please also review the below document about DS optimisation? https://www.gooddata.com/docs/cloud/connect-data/performance/ Do you see any space for improvement in your data source?
t
we don't really have the same issues on the visualization side of things, it's only the LDM that's catastrophically poor performance
✅ 1
when can I expect further info from the l2 team? We're blocked on client facing work with this issue
d
Hello Thomas! I’m from L2 team and I’m currently looking at this. I’ll be back with more news soon.
@Thomas may I know what node types are used for the Redshift instance?
r
Hi @Daniela Salmeron, thanks for checking this, we will share the details you requested via DM shortly
Done, please let us know if you require other information
d
Thanks! I sent you a follow-up question 🙂
@Thomas We would like to know what happened to the Redshift cluster on 27.03. from 14:20 CET to 14:25 CET in terms of executed queries. It can be done either via AWS web UI or via query similar to this one:
Copy code
select *
  from SYS_QUERY_HISTORY sqh
 where sqh.start_time > '2025-03-27 21:00:00'::timestamp
   and sqh.start_time < '2025-03-27 21:25:00'::timestamp
 order by start_time desc;
It will list SQL queries executed in defined time frame. What to look for: • Confirmation there are 3 queries against metadata taking about 30 seconds or longer. • How does the SQL queries look like? What are the explain planes and waiting times? • Is there any other heavy traffic against the cluster? Please send it to me once you have it. Thanks!
r
Hi @Daniela Salmeron, Given the volume of queries happened in those 5 minutes, could you help us narrow the search by providing details of what the query text might contain?
d
Hello! We are trying to check for queries that took more than 30 seconds or longer. Therefore, can you try this?
Copy code
select *
  from SYS_QUERY_HISTORY sqh
 where sqh.start_time > '2025-03-27 21:00:00'::timestamp
   and sqh.start_time < '2025-03-27 21:25:00'::timestamp
   and datediff(ms, start_time, end_time) >=100    
 order by start_time desc;
👀 1
r
Thanks Daniela, I've now sent you the queries via DM
Tagging @Rajat who is helping us gather the redshift information
👋 1
d
Thanks! we are checking it!
🙌 1
r
Hi @Daniela Salmeron, any news? We would appreciate any workaround in the meantime as well, as we're effectively blocked from making any modification to the ldm
d
Hello, May I just get your confirmation if besides blocking the LDM modification if this is also slowing down any report computation?
r
Hi Daniela, I heard from some (not all) of the customers that the data loads slowly in dashboards and visualisations, and I can reproduce it myself when creating certain reports. Not sure if this is caused by the same issue.
d
Thanks! I’m currently discussing this internally. I’ll have more news very soon! So far, we are seeing an issue with the newest Redshift JDBC driver, I’ll come back with more details
🙌 1
r
I've now shared the responses to the question you had via DM, Daniela.
d
Thank you!
Hello! I just sent you 2 queries via DM, please run them and let me know about he execution times. This will help us identify the cause and the next steps. Thanks!
✅ 1
r
Thanks Daniela, both of these queries were quite quick to run
d
Can you please let me know the execution times for each? 🙂 Thanks!
@Rajat @Raffaella Gozzini May I know if we could get the execution times for each? Thank you!
👀 1
r
Hi Daniela, sorry just saw this, getting those info, it takes few seconds sometimes to execute both those queries.
Currently querying to get more details when running that query
👍 1
r
Hi @Rajat did you manage to share this info?
r
Yes, I sent results to Daniela.
r
Thanks Raj. Daniela, looking forward to your update. Thank you
d
Hello! After checking the queries sent, we are seeing that the times are a bit different on our side. Therefore, we are checking in our own AWS Redshift on how could this be. In the meantime, may I know if it would be possible on your side to update the LDM via API? https://www.gooddata.com/docs/cloud/api-and-sdk/api/api_reference_all/#tag/LDM-Declarative-APIs I understand that via UI is the preferred way, however, we are still investigating and we don’t want to block you any longer.
r
Hi Daniela, for compliance with own internal process guidelines, we can update the LDM via API for test environments but we need to use the UI for changes to our root workspace or directly on customer workspaces.
Hi @Daniela Salmeron, any news?
d
Hello! Sorry for the radio silence! We have been testing better ways to scan the LDM. Today, we will release the fix. I’ll let you know once is in Production 🙂
Hello! I’m happy to let you know that the fix is in production. Please double check on your side and let me know. Thanks!
🌟 2
Hello @Raffaella Gozzini, May I know if you tested it? How is it looking on your side? Thanks
r
Hi @Daniela Salmeron we tested this and I am glad to confirm the fix seems to be working! Thank you!!
d
Glad to hear! Thanks for the confirmation 😊 Have a nice day
4 Views
Open in Slack
PreviousNext
Powered by