Dataflow を使用したサーバーレスのデータ処理 - Apache Beam と Dataflow を使用して ETL パイプラインを作成する（Python） のレビュー

VERY BAD>> I am unable to run my jobs are with DataflowRunner as I am always getting resources contraints errors.. job is not able to spn up us-cerntral1 region.. I am getting same error in all labs which requires to submit jobs on dataflow. I am able to run with DirectRunner. Please help in this as I have spent too many hours but end up getting same error again and again

Gurram Mallikarjunarao · 約11時間前にレビュー済み

couldnt finisih it lots of errors in the step by step or resources available

Petersen Sebastián · 約15時間前にレビュー済み

couldn't finish because dataflow job could get the resources to actually run. Stupid and a waste of my time. still can't run due to limited resources

Wilson Tyler · 約21時間前にレビュー済み

couldn't finish because dataflow job could get the resources to actually run. Stupid and a waste of my time.

Wilson Tyler · 約24時間前にレビュー済み

1. There are 2 issues running the lab: - lab 1: apache_beam.io.filesystem.BeamIOError: Match operation failed with exceptions {'gs://qwiklabs-gcp-04-9d1f7241cd59/events.json': RefreshError(TransportError("Failed to retrieve https://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true from the Google Compute Engine metadata service. Compute Engine Metadata server unavailable. Last exception: HTTPSConnectionPool(host='metadata.google.internal', port=443): Max retries exceeded with url: /computeMetadata/v1/instance/service-accounts/default/?recursive=true (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate - lab 2 Startup of the worker pool in us-central1 failed to bring up any of the desired 1 workers P.S. Not working http.client transport only supports the http scheme, httpswas specified P.S.S. export GCE_METADATA_MTLS_MODE=none resolves an issue with certificate. But still insufficient resources to run the job.

Prudyvus Igor · 1日前にレビュー済み

Cezar Junior Luis Antonio · 1日前にレビュー済み

Meng Lingmin · 2日前にレビュー済み

Meng Lingmin · 2日前にレビュー済み

Poor instructions - terrible!

Coombs Leighton · 3日前にレビュー済み

My conclusion from this lab is that I should not use or depend on this type of platform for production loads. It if does not even work in a controlled lab, then production is a no-go! ERROR:apache_beam.runners.dataflow.dataflow_runner:2026-04-01T09:25:07.346Z: JOB_MESSAGE_ERROR: Startup of the worker pool in us-central1 failed to bring up any of the desired 1 workers. This is likely a quota issue or a Compute Engine stockout. The service will retry. For troubleshooting steps, see https://cloud.google.com/dataflow/docs/guides/common-errors#worker-pool-failure for help troubleshooting. ZONE_RESOURCE_POOL_EXHAUSTED: Instance 'my-pipeline-1775035379230-04010223-mm24-harness-86c2' creation failed: The zone 'projects/qwiklabs-gcp-02-e142b19585b8/zones/us-central1-a' does not have enough resources available to fulfill the request. Try a different zone, or try again later.

Wikström Mikael · 3日前にレビュー済み

Cezar Junior Luis Antonio · 3日前にレビュー済み

In the Python script, allow users to pass alternative machine types, so there is no possibility of being unable to complete the lab due to insufficient resources being available (for N1?)

Vernieuwe Felix · 3日前にレビュー済み

1. There are 2 issues running the lab: - lab 1: apache_beam.io.filesystem.BeamIOError: Match operation failed with exceptions {'gs://qwiklabs-gcp-04-9d1f7241cd59/events.json': RefreshError(TransportError("Failed to retrieve https://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true from the Google Compute Engine metadata service. Compute Engine Metadata server unavailable. Last exception: HTTPSConnectionPool(host='metadata.google.internal', port=443): Max retries exceeded with url: /computeMetadata/v1/instance/service-accounts/default/?recursive=true (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate - lab 2 Startup of the worker pool in us-central1 failed to bring up any of the desired 1 workers P.S. Not working http.client transport only supports the http scheme, httpswas specified P.S.S. export GCE_METADATA_MTLS_MODE=none resolves an issue with certificate. But still insufficient resources to run the job.

Prudyvus Igor · 3日前にレビュー済み

1. There are 2 issues running the lab: - lab 1: apache_beam.io.filesystem.BeamIOError: Match operation failed with exceptions {'gs://qwiklabs-gcp-04-9d1f7241cd59/events.json': RefreshError(TransportError("Failed to retrieve https://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true from the Google Compute Engine metadata service. Compute Engine Metadata server unavailable. Last exception: HTTPSConnectionPool(host='metadata.google.internal', port=443): Max retries exceeded with url: /computeMetadata/v1/instance/service-accounts/default/?recursive=true (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate - lab 2 Startup of the worker pool in us-central1 failed to bring up any of the desired 1 workers P.S. Not working http.client transport only supports the http scheme, httpswas specified

Prudyvus Igor · 3日前にレビュー済み

Cezar Junior Luis Antonio · 4日前にレビュー済み

Ferrari Charles · 4日前にレビュー済み

Too much confuse the Python file and Instructions were not clear to proceed and modify the files. The code is running with errors. Need support.

Naga Venkateswarlu Kuriseti · 4日前にレビュー済み

leela jogeswara rao konda · 4日前にレビュー済み

Monge Jose Jorge Alberto · 7日前にレビュー済み

There is currently an issue with google.auth library (https://github.com/googleapis/google-cloud-python/issues/16090) that prevents me from finishing the lab - took a long time to find the root couse and I run out of time. Additionally, the worker zones and machine types have to be specified because without them the dataflow cannot spawn workers and jobs keep failing.

Slys Przemyslaw · 7日前にレビュー済み

1. There are 2 issues running the lab: - lab 1: apache_beam.io.filesystem.BeamIOError: Match operation failed with exceptions {'gs://qwiklabs-gcp-04-9d1f7241cd59/events.json': RefreshError(TransportError("Failed to retrieve https://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true from the Google Compute Engine metadata service. Compute Engine Metadata server unavailable. Last exception: HTTPSConnectionPool(host='metadata.google.internal', port=443): Max retries exceeded with url: /computeMetadata/v1/instance/service-accounts/default/?recursive=true (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate - lab 2 Startup of the worker pool in us-central1 failed to bring up any of the desired 1 workers

Prudyvus Igor · 7日前にレビュー済み

Baran Bartłomiej · 7日前にレビュー済み

Ennoconn Alice Wu 吳馨妤 · 10日前にレビュー済み

not enough recources to complete the lab

van Hoek Stef · 10日前にレビュー済み

Shaw Sriyansh · 10日前にレビュー済み