Hi tried to connect to our big query and got the following message when trying to scan the data:

Hi Chilik, It seems like the bigquery dataset specified in the datasource configuration does not exist - we can see following message in our logs (propagated from the bigquery):

message=com.google.cloud.bigquery.BigQueryException: Not found: Dataset ***,<no detail>

Admittably the 500 error is not very user friendly - we'll get it fixed.

Thank you @Boris could you suggest why this may happen? could this be related to the permission sets we defined in the big query?

Hi Chilik, it's possible that the permissions affect this. Basically, when you creating a connection to bigquery, you upload the service key, which set ups the connection and then you need to specify the dataset in which the tables will be scanned. What happened is that the dataset you specified does not exist, or the service account used for the connection does not have access to it or it could be blocked from the ip addresses which our system uses (mentioned here). It is also case sensitive, so it might be a typo.