Hey <@U04PGCQB7K6> and <@U05KXRXS9FU>, Could you b...
# gooddata-cloud
i
Hey @Steve Fox and @Sean Barry, Could you be so kind and move your discussion under one thread, please? I will review the health of the Cloud/Auth0 in the meanwhile.
s
Will do.
We use our own authentication, so its not Auth0,
1
Did anything change with permissions?
because we're seeing weird issues with a user that used to have admin priveleges and now that user only shows 1 workspace
s
We’re dead in the water durning major demos, can we make sure this is critical
s
Not cool
Submitted a ticket as well to support
i
Thank you both! It might be related to today’s Release.
s
Yeah there are no details in those notes, already looked there
i
Yes, I can see your ticket as well. Please give me a minute to check the situation internally - I am really sorry for your troubles here.
Thank you for your patience, gentlemen. Unfortunately, I can confirm there is really an issue tied to the Release. Our engineers are already working on the prompt fix. I will keep you posted and once again, I apologize for any inconveniences caused by this!
s
Do we have an approximate eta?
i
Not yet, unfortunately.
s
and definitely keep us posted as @Sean Barry mentioned, its not a great time for this issue for us.
1
Do we know what hte issue is?
i
The issue seems to be related to our Meta Data Database. I understand the urgency of the situation and I’ve also passed the info to our engineers on duty.
s
I'm also seeing we can't seem to find any of our admin users in our production instance
when i query for all usera with their usergroups -> no one is an admin
this related?
0 admins
i
Most probably it is, we can see also some performance issues tied to Insight/Dashboard executions.
s
yeah this is like people are either missing or not included in the groups that should be there.
almost seems like our whitelabelling /auth settings aren't working for our production instance.
When did the release notice get posted? I'm following hte channels and did not see a notice about an upcomming release. Woke up this am to the email about the release happening.
i
I know it doesn’t help much in this unlucky situation, but the note was published two days ago: Oct 3, 2023, 125211 PM CEST.
s
Ah, I was only subscribed to the release notes section, not the service announcements.
at least i'll see both form now on
🙌 1
i
We are about to announce the issue publicly soon as well.
s
Are we the only ones who have this issue?
Or the only ones who've raised it so far
i
Unfortunately, the issue is affecting (impact might be different) all customers on IAD1 Datacenter in us-east-1 region. We’ve just published the Announcement as well.
s
Well at least I can confirm i got the notice from the announcement.
1
Not sure we knew we were in that data center until now 🙂
f
@Ivana Gasparekova can you confirm if our issue is related to release as well? I don't see our issue being reported by anyone else, so asking to confirm. https://gooddataconnect.slack.com/archives/C04S1MSLEAW/p1696517549557779
i
@Filip Charikov - yes, it is. We are seeing some long running queries, which are affecting the overall performance as well. Therefore, no action needed from your end. Please accept my apology for this unpleasant situation.
s
just to confirm, our issue not being operational at all is a performance issue?
i
Yes, Steve, I am aware of it. As I said, the impact is not the same for every organization. Fixing the issue is our highest priority right now!!
s
All good, just the announcement and the discussion so far seems to be about long running queries. I wish that was our issue at this point 🙂
i
I feel you, Steve. 😞
s
Is it possible our admin usergroup is gone?
in production?
i
I hope it will get “recognized” shortly again. We are just about to revert some of the recent changes. Please stay tuned!
s
Good to hear
i
Some changes were reverted already - could you check if you can see any progress on your end, please?
s
Quick test it appears back up
🙌 1
i
So glad to hear it, @Sean Barry.
f
No changes on our end
s
@Ivana Gasparekova I'm still not seeing the admin group on users.
Like my user should have the admin group but when I query the api for my user the admin group does not return.
i
Sadly, it looks like it fixed just part of the issue.
s
Is that new/expected?
i
@Steve Fox - can we continue in the ticket, please?
s
Ok.
i
@Filip Charikov - can we create a follow-up ticket on your behalf as well, please?
s
We’ve got a lot of environments and a lot of test platforms. If you need us to check on things, we’d be happy to help with that. Being honest, this happened the the absolute worst time for us. We’ve seen this happen a number of times this year, but weren’t yet live. We still have security settings that are just not correct.
i
@Ivana Gasparekova Where should that ticket be created?
@Filip Charikov - can we create a follow-up ticket on your behalf as well, please?
@Ivana Gasparekova Where should that ticket be created?
I've created a ticket, it's number is #117199
j
@Igor Cherkasov the ticket number is #117195
1
🙌 1