We seek a Lead Data Engineer with a passion for baseball and technology to lead the development of timely and reliable data pipelines. Your work will deliver the data that informs the decisions that build a sustainable winning team in Miami.
Key Responsibilities:
● Serve as the tech lead for the Marlins’ Data Engineering team.
● Lead the design, development, documentation, and maintenance of schemas and ETL pipelines for internal databases and data warehouses that scale and adapt to future use cases.
● Break down complex data engineering projects into actionable work plans including proposed task assignments with clear design specifications.
● Implement and test collection, mapping, and storage procedures for secure access to internal and external data sources.
● Assess, provision, monitor, and maintain the appropriate infrastructure and tooling to automate and execute data engineering workflows.
● Develop algorithms for quality assurance and imputation to prepare data for exploratory analysis and quantitative modeling.
● Coordinate with data providers around planned changes to data feeds.
● Research, design, test, and implement generalizable software architectures for data ingestion, processing, and integration and guide organizational adoption and strategies for infrastructure maintenance and data-related cost savings.
● Define and encourage team-wide adoption of data engineering standards.
● Effectively communicate complex technical concepts to both internal and external audiences.
● Assist with recruiting and outreach for the engineering team, including building a diverse network of future candidates, and provide guidance and technical mentorship for junior engineers.
● Fulfill other related duties and responsibilities, including rotating platform support.
Qualifications:
● Significant experience in back-end software design and development.
● Experience with ETL architecture and development in a cloud-based environment.
● Fluency in SQL and an understanding of database and data warehousing technologies.
● Proficiency with Python (preferred), Scala, and/or other data-oriented programming languages.
● Experience with automated data quality validation across large data sets.
● Familiarity working with virtual machines in a cloud environment.
● Strong software engineering and problem-solving skills.
● Expertise developing complex databases and data warehouses (e.g. BigQuery, RedShift, Snowflake) for large-scale, cloud-based analytics systems.
● Experience with task orchestration and workflow automation tools such as Airflow.
● Proficient in designing, deploying, and managing containerized applications with tools such as Docker and Kubernetes.
● Ability to successfully coach junior engineers to grow in their own careers.
We are an equal opportunity employer and all qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, sex, sexual orientation, age, disability, gender identity, marital or veteran status, or any other protected class.
