Data Engineer (Cloud Big Data)
About the Job
Working under direction of Data Architects and Technical Managers, this position is responsible for building data pipelines into and out of a data lake using state of the art technologies. This includes the ingestion and transformation of data that provides critical information to the enterprise and business partners. This position can be in the Atlanta or New York City office.
- Create solutions to transform data from various sources and load it into Snowflake to create a data lake
- Create and maintain transformations to summarize/aggregate data and load this so users can consume this data using various BI/Analytics tools
- Develop and maintain standards for administration and operation including the scheduling, running, monitoring, logging, management of errors, recovery from failures, and validation of outputs
- Contribute to the project planning process by estimating tasks and deliverables
- Work closely with Revenue Analytics team members to understand user requirements
- Be at the cutting edge of utilizing data about consumers in the media industry to improve audience experience
- Minimum 3-4 years solid experience writing Object-oriented programs in Python
- Minimum 2-3 years solid experience writing SQL
- Experience in working with Linux environments and Unix Shell Scripting
- Understanding of ETL processes and design
- Basic understanding of statistics
- BS or higher in Computer Science, Mathematics, MIS, business, or equivalent education / training / experience
The Extra Credit
- Experience with Apache Airflow or similar job scheduling application.
- Knowledge and working experience with AWS such as S3, Lambda, EC2, Kinesis, etc.
- Knowledge of Luigi, AWS Glue, AWS Athena, and Apache Spark.
- Experience with "big data" platforms such as Snowflake, Hadoop, Hive, Presto or cloud-based tools such as Amazon Redshift, Google BigQuery
- Experience working with REST and SOAP APIs
- Specific application experience with media, Web Analytics and consumer data systems a plus
- Experience with BI tools such as Looker and Tableau
- Paid time off every year to volunteer.
- Access to well-being tools, resources, and freebies
- 2018 Best Company for Working Mothers
- 2018 Best Company for Dads
- Part of the Time Warner family of powerhouse brands like HBO, Cartoon Network, Adult Swim, TNT, and TBS.