Hi everyone I had problems loading data for a week, it took a very long time and sometimes there were errors in the process this happened just a few days ago This problem has not occurred before and I have not found a solution for this process Thanks

Hi Adel, can you please send me the related Request ID or Error ID you’re seeing the log output and I will check our logs for you and try and see if we can get more info on this for you

Sorry Michael I've deleted the previous error, but I'm running the load process and it's still taking a long time. Request id: "data_load_LeFHrVsYsn_OrX5AWQgIRMKtfhUm2ogSUexD6sRcpGkiRJjC00Jw7"

When you say “Long time” can you please provide more details?

Previously the process was still running smoothly and I didn't change anything in the database Today I didn't get a successful return and it's still running

From our logs I can see that this process started at: 09:53 (CET) - Let’s see how this one finishes and if it finishes with an error, please send me the related details from the logs and I will check our internal logs for you. The longer loads times may just be caused by are larger amount of data being upload into your workspace(s) as you mentioned you were having issues this week. But we will see how this ETL finishes and I will investigate further.

Ok Michael, Thank you for the help

It seems that the issue relates to BigQuery and it is closing the connection on their side due to the size of the data load, I see the table is being loaded in “full” mode. To tackle the issue, I suggest looking into switching the load mode to “incremental”, loading the data in smaller chunks, setting a different ETL schedule for the problematic table, or setting the parameter

GDC_DOWNLOAD_PROJECTS_CHUNK_SIZE

to

1

which will enable each task to only download data of 1 project at a time (instead of proceeding in chunks of 4). 2 or 3 may work as well. This is further described in Best Practises.