BigData Expert/Developer - Locals only - Redmond, WA - Full Time

WA · 120 months ago

About the job

Job Description Ø Build the Event Hubs integration with Service Fabric micro services implementation. Streaming the processed files from blobs into EH for downstream processing. Ø Anonymized files (~1000 of them and to a size of ~GB) will be given as input Ø Service Fabric code portion will be provided. Ø Build the Spark processing reading off EventHubs, implementation in either Python or Scala would suffice. Ø Look at the caching needs; leverage .cache to retain appropriate results from Spark ‘Actions’ in Spark executors Ø Our team will evaluate a set of data store that would be a landing spot post Spark – Blobs being a required one. We will pick 1 or 2 from this list -- SQL DW, Azure SQL...