Serverless Data Processing with Dataflow - Writing an ETL pipeline using Apache Beam and Dataflow (Java) Ulasan

Nadine E. · Diulas hampir 4 tahun lalu

just 2 things I need to comment on this lab: first: the pipeline create the logs table only if the line "CREATED_IF_NEEDED" is deleted secondly: the schema.json copied on GCS is not detected on the lab.

Mohammed H. · Diulas hampir 4 tahun lalu

Larry F. · Diulas hampir 4 tahun lalu

Jéssica Isri D. · Diulas hampir 4 tahun lalu

ide problem

Manoj K. · Diulas hampir 4 tahun lalu

Manoj K. · Diulas hampir 4 tahun lalu

Eduardo D. · Diulas sekitar 4 tahun lalu

Ákos M. · Diulas sekitar 4 tahun lalu

There is something wrong with the 'Creating a JSON schema file' check. The file is correctly being generated and uploaded into the bucket but the check still doesn't shows up. Unable to finish due to this bug...

Eliseo F. · Diulas sekitar 4 tahun lalu

Dilu M. · Diulas sekitar 4 tahun lalu

Francesco C. · Diulas sekitar 4 tahun lalu

Very difficult to understand the lab......a video might be better instead of the written format

Bhavin K. · Diulas sekitar 4 tahun lalu

Marcelo S. · Diulas sekitar 4 tahun lalu

Sebabrata G. · Diulas sekitar 4 tahun lalu

It's a quest and not a lab. There is a bug in check performed for "creating a JSON schema file"

Onkar K. · Diulas sekitar 4 tahun lalu

It's a quest and not a lab. There is a bug in check performed for "creating a JSON schema file"

Onkar K. · Diulas sekitar 4 tahun lalu

It's a quest and not a lab

Onkar K. · Diulas sekitar 4 tahun lalu

Nobuhiro O. · Diulas sekitar 4 tahun lalu

Needs to be debugged, there is a dependency issue. I am struggling since not very handy with coding, especially Java. Also Qwiklabs team is taking a lot of time (days to resolve) [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project advanced-dataflow-lab-1-lab: Compilation failure: Compilation failure: [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[19,23] package com.google.gson does not exist javac -classpath <LOCATION_TO_GSON JAR FILE> *.java

Vikas W. · Diulas sekitar 4 tahun lalu

Mohamed A. · Diulas sekitar 4 tahun lalu

Needs to be debugged

Vikas W. · Diulas sekitar 4 tahun lalu

Bug in description, bugs in scripts, and again solution does not include first stage with dump using Gson. Very poorly prepared.

Mariusz S. · Diulas sekitar 4 tahun lalu

Hoe Kit C. · Diulas sekitar 4 tahun lalu

[INFO] ------------------------------------------------------------- [ERROR] COMPILATION ERROR : [INFO] ------------------------------------------------------------- [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[27,26] cannot find symbol symbol: class Logger location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[33,38] cannot find symbol symbol: class DataflowPipelineOptions location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[65,19] cannot find symbol symbol: class PipelineResult location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[27,39] cannot find symbol symbol: variable LoggerFactory location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[46,27] cannot find symbol symbol: variable PipelineOptionsFactory location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[68,9] cannot find symbol symbol: class Pipeline location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[68,29] cannot find symbol symbol: variable Pipeline location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[69,16] cannot find symbol symbol: method setJobName(java.lang.String) location: variable options of type com.mypackage.pipeline.MyPipeline.Options [INFO] 8 errors [INFO] ------------------------------------------------------------- [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 6.548 s [INFO] Finished at: 2022-05-01T12:01:08+00:00 [INFO] Final Memory: 39M/58M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project advanced-dataflow-lab-1-lab: Compilation failure: Compilation failure: [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[27,26] cannot find symbol [ERROR] symbol: class Logger [ERROR] location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[33,38] cannot find symbol [ERROR] symbol: class DataflowPipelineOptions [ERROR] location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[65,19] cannot find symbol [ERROR] symbol: class PipelineResult [ERROR] location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[27,39] cannot find symbol [ERROR] symbol: variable LoggerFactory [ERROR] location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[46,27] cannot find symbol [ERROR] symbol: variable PipelineOptionsFactory [ERROR] location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[68,9] cannot find symbol [ERROR] symbol: class Pipeline [ERROR] location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[68,29] cannot find symbol [ERROR] symbol: variable Pipeline [ERROR] location: class com.mypackage.pipeline.MyPipeline [ERROR] /home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs/src/main/java/com/mypackage/pipeline/MyPipeline.java:[69,16] cannot find symbol [ERROR] symbol: method setJobName(java.lang.String) [ERROR] location: variable options of type com.mypackage.pipeline.MyPipeline.Options [ERROR] -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project advanced-dataflow-lab-1-lab: Compilation failure at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106) at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863) at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288) at org.apache.maven.cli.MavenCli.main(MavenCli.java:199) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) at java.lang.reflect.Method.invoke(Method.java:508) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) Caused by: org.apache.maven.plugin.compiler.CompilationFailureException: Compilation failure at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:1220) at org.apache.maven.plugin.compiler.CompilerMojo.execute(CompilerMojo.java:187) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207) ... 20 more [ERROR] [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException theia@eb75c3cda9fd:/home/project/training-data-analyst/quests/dataflow/1_Basic_ETL/labs$

Nishanth V. · Diulas sekitar 4 tahun lalu

Requires more time to finish for those new to Beam and or Java

Prateek P. · Diulas sekitar 4 tahun lalu