Senior Data Engineer (FinTech | Market Data)
Project Overview
We are looking for an experienced Senior Data Engineer to join a project for a large US-based global investment and technology company operating in the Investment Management / FinTech domain. This is a full-time, on-site position based in Dallas, TX.
The engineer will work with large volumes of financial and market datasets, including equities, macroeconomic data, and other non-standard data sources used by quantitative researchers, traders, and analysts. The role provides an opportunity to work with complex data pipelines that have a direct impact on investment research and trading-related systems, within a strong engineering environment with high requirements for data quality, timeliness, and reliability.
Key Responsibilities:
- Own data pipelines end-to-end, including ingestion, processing, validation, and publication.
- Build and maintain batch and streaming data pipelines.
- Implement reliable processing for idempotency, backfills, late-arriving data, and partitioning.
- Develop automated data quality checks covering completeness, timeliness, anomalies, and consistency across sources.
- Monitor data pipelines and configure effective alerting.
- Investigate and resolve data quality and pipeline reliability issues.
- Optimize pipeline performance and resource usage.
- Onboard new datasets and data providers.
- Work with large volumes of financial and market-related data.
- Communicate with data consumers about incidents, data issues, changes, and availability.
- Collaborate on the development of APIs and internal libraries for data access and distribution.
- Contribute to improving reliability, maintainability, and observability of the data platform.
Required Skills & Experience
- 5+ years of professional experience in Data Engineering.
- Strong practical experience with Python.
- Strong knowledge of SQL.
- Strong hands-on experience with Apache Spark.
- Experience with Hadoop / Hive or modern lakehouse formats such as Delta Lake or Apache Iceberg.
- Experience building and maintaining batch and streaming pipelines.
- Strong understanding of:
- idempotent data processing;
- backfills;
- late-arriving data;
- partitioning strategies.
- Experience implementing automated data quality checks.
- Experience with monitoring and alerting for production data pipelines.
- Experience working with large volumes of data.
- Strong understanding of reliable data ingestion and publication processes.
- Strong analytical and troubleshooting skills.
- High attention to detail and data correctness.
- Strong ownership and ability to independently manage data engineering tasks.
- Strong communication skills when working with data consumers and technical stakeholders.
- Valid US work authorization.
- Ability to work on-site full-time, Dallas, TX .
Nice to Have
- Experience working with market data, including equities, macroeconomic data, or market data vendors.
- Experience with Kafka or other streaming technologies.
- Experience with Airflow or another workflow orchestration platform.
- Experience with Snowflake or Databricks.
- Experience developing APIs or reusable libraries for data distribution.
- Experience with Java or Scala.
- Previous experience in Investment Management / Financial Services.