Hello GD team, one of my colleagues who's been added to a user group, and that user group has been granted

Manage

access on a data source. He then attempted to upload additional data, but faced with this error. If it helps, his user is Nick Paul and he's been added to the

ts-internal-se-group

, and he's working on the

ts-sd1-csv-ds

data source as seen in screenshot. Pls kindly assist, thank you!

Hello Alson, could you please confirm the user/userID and let us know if they have issues accessing the workspace or just these csv files?

Hi @Joseph Heun thanks for your reply. His user is

nickpaul

he has no issues accessing the workspace. He is attempting to upload additional files

Could you please confirm fi the user is an admin? I believe that would be the only other restriction here. How was @Zhi Yi Yeo able to resolve this?

Had to promote him to admin in the meantime to quickly allow him to upload some files, else the problem still remains

Can you please check in the UI that the user group has permission to manage the data source - You can find an example screenshot below. More details on this can be found here: https://help.gooddata.com/doc/growth/en/data-integration/data-preparation-and-distrib[…]data-loading-processes/share-a-data-source-with-other-users/

Hi @Michael Ullock yes. As mentioned in the initial message, the group has Manage access to this data source

Hi Alson, I will need to impersonate the user and check the settings from within the org. I will get back to you with my findings shortly

Yes sure please do

Hi Alson, I think we may have overlooked this. the user group he should be added to is ts-internal-ds-group, as that is that group that has access to the data source.

Hi @Joseph Heun when @Zhi Yi Yeo was in that group, he encountered his error as well. And the

ts-internal-se-group

has

Manage

access to the data source (that Nick is in)

the above image shows the groups that have access to ts-sd1-csv-ds. I don't see that ts-internal-se-group has access there

how is it that this group doesnt have access though, cause when I looked at this group's accesses, it does show the data source and workspace access

indeed. seems like a misbehavior. Could you please try and add the group to the data source and see if that allows the group access?

hmm, i am opening up the data source's

Permissions

tab now. and am seeing these

So I've just added him as a solo user to the data source, and now it shows up when I impersonate his user so it appears he has access to it. However, something does seem off with the groups here

Sure, I create the users via calling an API that's written in Python.

def create_user(user, headers):
    """Create a user in the GoodData system."""
    create_user_endpoint = settings.GOODDATA_ENDPOINTS["CREATE_USER"]
    try:
        payload = create_user_endpoint.payload.format(  # type: ignore
            user_id=user.id,
            first_name=user.first_name,
            last_name=user.last_name,
            email=user.email,
            authentication_id=user.user_id,
            group_id=settings.TS_MT_DEV_GROUP_ID,
        )
        response = requests.request(
            method=create_user_endpoint.method,
            url=create_user_endpoint.url,
            headers=headers,
            data=payload,
        )
        response.raise_for_status()
        logger.success(f"Successfully created user: {user.id}")
        return response.json()
    except requests.exceptions.RequestException as e:
        logger.error(f"Error creating user {user.id}: {e}")
    except Exception as e:
        logger.error(f"Unexpected error while creating user {user.id}: {e}")

And if you're wondering about what that

CREATE_USER

key-value pair is:

GOODDATA_ENDPOINTS: Dict[str, GoodDataAPIEndpoint] = {
        "GET_USERS": GoodDataAPIEndpoint(
            method="GET",
            url=f"{GOODDATA_HOST_URL}/api/v1/layout/usersAndUserGroups",
        ),
        "CREATE_USER": GoodDataAPIEndpoint(
            method="POST",
            url=f"{GOODDATA_HOST_URL}/api/v1/entities/users",
            payload="""
                {{
                    "data": {{
                        "id": "{user_id}",
                        "type": "user",
                        "authenticationId": "{authentication_id}",
                        "attributes": {{
                            "firstname": "{first_name}",
                            "lastname": "{last_name}",
                            "email": "{email}",
                            "authenticationId": "{authentication_id}"
                        }},
                        "relationships": {{
                            "userGroups": {{
                                "data": [
                                    {{
                                        "id": "{group_id}",
                                        "type": "userGroup"
                                    }}
                                ]
                            }}
                        }}
                    }}
                }}
        """,
        ),

Once the user is created via code, then I will go into the GD UI and assign the user into the group

Hi Alson, could you add joseph.heun@gooddata.com to the org and then the group in the same way? I'd like to see the behavior if you do it now

Hi Joseph, unfortunately Im not sure if this is possible, because our SSO is tied to our Google account, and that means we can only allow users with

<mailto:xx@terrascope.com|xx@terrascope.com>

email domain to log in. I could create a user without any SSO authentication, and generate an API token for you though. Not sure which do you think is better. Alternatively, would you require Nick to provide you with trace ID instead?

could you confirm if Nick is able to work with the CSV now?

will need him to confirm on his side. Not sure if not doing anything on either end will work though. @Zhi Yi Yeo if you happen to be online (not urgent), i've removed your admin access, could you help to check if you still face the same issue on your side?

so it seems @Zhi Yi Yeo does not have access to the data source anymore, and I only see them in the DS group

yes indeed, but when he attempts to access the

ts-mtdev-csv-ds

, he should be able to

Manage

it, as that group has such an access right?

That should be fine according to the groups. Let me impersonate his user and check the behavior

yes, Zhi Yi is in DS group, to access

ts-mtdev-csv-ds

. Nick is in SE group, to access

ts-sd1-csv-ds

Everything seems to be working as expected now. I'll need to test it once you add a new user, as I cannot find where the bottleneck would be in this instance at all

Okay sure. @Nick Paul @Zhi Yi Yeo could you both assist to check if it's working as expected pls?

hi @Alson Yap @Joseph Heun, I am still getting an error when trying to upload data

@Joseph Heun could you pls check on your side?

I'm still experiencing the same error that Zhi posted above

For GD folks - Do we need to get into a call to speak over this? Although I think the problem statement is clear enough and we just need to figure out why the backend is denying the user

Hey everyone. I wanted to let you know that we are escalating this to our L2 technical support team to look into this further for you. We hope to uncover the bottleneck in this soon, and will follow up with more details as they present themselves.

Ok thank you, pls let us know

Hi @Alson Yap and team, this is Francisco from the Technical Support Team. I’ve started my investigation of this issue. I have been able to reproduce this behavior on my end - indeed, it looks like

MANAGE

permissions to a CSV Data Source via userGroups or even directly assigned to a User is not sufficient to allow users to modify the data source (in this case: add or update the CSV files). I will continue to investigate this in more detail, but for now I would recommend adding your users to the adminGroup as a workaround, if they need to make changes to Data Sources. From my tests, I haven’t been able to grant the correct permissions any other way.

Hi Francisco, thank you for looking into this and confirming the issue. Pls do keep us posted when you have new findings

Hi @Alson Yap I have decided to escalate the issue to our engineers, as this is not the preferred behavior in this case. I will let you know once once we’ve heard back from them!

Oh okay I see. Thanks for following up on this @Francisco Antunes, pls let me know when this is available soon. I hope the other data sources should behave as expected, such as Flex Connect?

This should not affect any other Data Sources! To be a bit more technical, the problem was in the endpoint that is specifically called when users attempt to upload new data into the CSV Data Source, that uploads data to the Staging Area. That specific endpoint exists only for holding the CSVs before they can be imported, and as it turns out it was only available for users with

org.MANAGE

rights (that is, domain admins). The change will allow users with

MANAGE

permissions on the CSV data source to access that endpoint, too, fixing the problem. We do not expect an impact on any other features.

Confirming this fix has corrected my ability to upload custom data sources! 🎉

Thank you @Francisco Antunes for the explanation and the update, glad to hear that this has been fixed 😄 For @Zhi Yi Yeo @Nick Paul I've removed the admin access now, pls do follow up here if there's further issues 🙏

can confirm that it is working for me now, thanks all!