Serverless Data Processing with Dataflow - Advanced Streaming Analytics Pipeline with Dataflow (Python) Reviews

Zelmar M. · Reviewed about 4 years ago

There are a couple of problems with this lab. First, when running the pipeline I got the following error: ValueError: GroupRecordsByDestinationAndShard: Unsafe trigger: `AfterProcessingTime(delay=120)` may lose data. Reason: MAY_FINISH. This can be overriden with the --allow_unsafe_triggers flag. I was able to fix this by changing this: trigger=AfterProcessingTime(120) to this: trigger=Repeatedly(AfterProcessingTime(120)) Second, we are asked to add the following to ConvertToCommonLogFn: yield beam.pvalue.TaggedOutput('unparsed_row', element.decode('utf-8')) However the lab code already includes the following in the previous line: element = element.decode('utf-8') If this isn't removed, we get the following error due to double-decoding: AttributeError: 'str' object has no attribute 'decode' [while running 'ParseJson/ParDo(ConvertToCommonLogFn)-ptransform-81']

Robert L. · Reviewed about 4 years ago

Hanh T. · Reviewed about 4 years ago

Edward U. · Reviewed about 4 years ago

Jack R. · Reviewed about 4 years ago

Thiago M. · Reviewed about 4 years ago

Krishna V. · Reviewed about 4 years ago

To complete this task is necessary install python 3.8 version

Giuseppe M. · Reviewed about 4 years ago

Maria I. · Reviewed about 4 years ago

Olga P. · Reviewed about 4 years ago

Jean-Edouard R. · Reviewed about 4 years ago

Eber D. · Reviewed about 4 years ago

Errors in the lab could not complete. Team is investigating the issue due to diff in python version.

Krishna V. · Reviewed about 4 years ago

Sascha D. · Reviewed about 4 years ago

The code installs python 3.9.2 which is not compatible with Dataflow. Please check

Nitin A. · Reviewed about 4 years ago

Mihir D. · Reviewed about 4 years ago

Mihir D. · Reviewed about 4 years ago

LACETERA C. · Reviewed about 4 years ago

Sachin K. · Reviewed about 4 years ago

Emilio José L. · Reviewed about 4 years ago

bert s. · Reviewed about 4 years ago

Muhammad S. · Reviewed about 4 years ago

Emilio José L. · Reviewed about 4 years ago

Jivesh S. · Reviewed about 4 years ago

Muchamad R. · Reviewed about 4 years ago