Data Engineer

We're seeking data engineers to build the backbone of Hedgineer's platform- scalable ETL pipelines and data infrastructure that power risk management, portfolio analytics, and AI capabilities for hedge fund clients. This role goes beyond traditional data engineering: you'll design and contribute to AI-powered systems that automate pipeline creation, optimise data workflows, and enable rapid deployment of custom data solutions. The ideal candidate combines strong data engineering fundamentals with genuine curiosity about AI- someone excited to explore cutting-edge approaches to code generation, pipeline orchestration, and intelligent automation.

Responsibilities:

• AI-Driven Automation: Design and implement AI systems that automate pipeline creation, data transformation logic, and workflow orchestrationpushing the boundaries of what's possible in data engineering.
• Pipeline Development: Build scalable, high-performance ETL pipelines that handle critical financial market data with reliability and efficiency.
• Event-Driven Architecture: Design event-driven ETL systems to support both real time and batch data processing needs.
• Intelligent Infrastructure: Experiment with and deploy AI agents, code generation tools, and LLM-powered solutions to accelerate data platform development.
• Analytics and Insights: Run analytics on large datasets to uncover insights, validate models, and propose new platform features.
• Data Modeling: Architect data structures for both analytical (OLAP) and transactional systems using appropriate database technologies.
• Cross-Team Collaboration: Partner with engineers across Bangalore and New York to maintain data integrity, security, and performance standards.

Requirements:

• A bachelor's or master's degree in computer science or a comparable subject.
• 3+ years of software engineering experience with focus on data engineering.
• Exceptional programming skills in Python and SQL.
• Expertise with data manipulation libraries (Pandas, Polars).
• Experience building production-grade ETL pipelines using orchestration tools (Airflow, Dagster, Prefect).
• Hands-on experience with DBT and DuckDB for data transformation and querying.
• Familiarity with event-based ETL architectures for data processing.
• Deep understanding of database technologies and when to use themexperience with Snowflake, MSSQL, Postgres, or similar.
• Passion for AI: Genuine curiosity about how AI can transform data engineering experience using AI tools in development workflows (Claude, GitHub Copilot, or similar).
• Strong problem-solving abilities and comfort handling ambiguity.
• Excellent communication skills for collaborating across distributed, cross functional teams.
• Experience building resilient data pipelines on Databricks using Delta Lake, Auto Loader, and workflow orchestration AI Implementation.
• Hands-on experience with AI agentic frameworks (Claude Code, LangGraph), LLM APIs, prompt engineering, or building AI-powered automation tools.
• Exposure to financial domain analytics or experience working with hedge funds/asset managers.
• Understanding of financial data models (positions, securities, market data, risk metrics).
• Experience with cloud platforms (Azure, AWS, GCP) and cloud-native data architectures.
• Familiarity with code generation, AI-assisted development, or building developer tools.
• Familiarity with data quality frameworks and monitoring systems.
• Strategic thinking with ability to contribute to product roadmap discussions Computer science, engineering, or related degree.