Big Data Lead

Big Data Engineer

Responsibilities/Duties

• Make major contribution to the implementation of our real time big data initiative
• Build and automate productized data processing pipelines in AWS big data platform
• Help improve our development process and standards through mentoring and leading-by-example
• Research and develop statistical learning models for data analysis
• Implement new statistical or other mathematical methodologies as needed for specific models or analysis
• Help define and implement data ingestion contracts
• Create data environment to support our data analytics, reporting and data science teams

Knowledge, Skills and Abilities

• In-depth understanding of modern big data technology, including Hadoop and Spark
• Data modeling and machine learning skills
• Knowledge of real time data streaming and aggregation architectural patterns and practice
• Proficient in programming languages such as Python, Scala and Java
• Familiarity with SQL as well as SQL databases

Education/Experience

• Bachelor’s Degree in computer science, data science or related fields
• Familiarity with agile developmental process
• Previous experience developing data product required
• Hands-on experience with real time data streaming, aggregation and presentation strongly preferred
• Previous experience with production ETL pipeline development required
• At least a years’ experience with AWS cloud or another cloud platform