Lead Machine Learning Engineer

Responsibilities:

• Will collaborate closely with our data science and ML teams across different cybersecurity domains to define ML infrastructure requirements and build critical data services.
• Can leverage MLOps best practices to design and develop scalable model training, evaluation, experimentation, and deployment workflows.
• Has extensive experience in ML training (local and distributed), feature extraction, and dataset creation.
• Is comfortable deploying software with CI / CD tools, including Jenkins, Harness, Terraform, etc.
• Is an expert at developing and deploying assets in the cloud, preferably AWS and Kubernetes, using IAC (infrastructure as code).
• Can build a workflow orchestration platform to be used by other developers.

Requirements:

• Has hands-on experience of 2+ years implementing data pipeline infrastructure for data ingestion and transformation near real-time availability of data for applications and ML pipelines.
• Has experience designing optimized solutions for ingestion, curation of large datasets.
• Has working knowledge of Data Lake technologies, data storage formats (Parquet, ORC, Avro), and query engines (Athena, Presto, Dremio), and associated concepts for building optimized solutions at scale.
• Maintains a proficient level in one of the following programming languages or similar- Python, Java, Go.
• Has experience with data pipeline tools (Flink, Spark, or Ray) and orchestration tools such as Airflow, Dagster, or Step Functions.
• Is an expert in implementing data streaming and event-based data solutions (Kafka, Kinesis, SQS/SNS or the like).

Skills

ML, flink, go, java, machine learning, python, ray, spark