Machine Learning Engineer — AI Architecture Research

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Machine Learning Engineer - AI Architecture Research based in South Africa.

This role focuses on researching and building next-generation AI model architectures that can move from experimental concepts to scalable production systems.

You will work at the intersection of machine learning research, model engineering, and real-world deployment.

The position offers the opportunity to challenge established architectural assumptions and explore alternatives to conventional Transformer-based designs.

You will design experiments, prototype new neural networks, and evaluate trade-offs across compute, memory, latency, and model performance.

The role involves close collaboration with inference and systems engineers to make research ideas efficient and deployable.

You will also contribute to research reproduction, benchmarking, technical exploration, and potentially open-source work.

This is an opportunity to have meaningful influence on AI architecture while working in a fast-moving, research-oriented environment.

Accountabilities:

• Research and develop novel neural network architectures, including alternatives or extensions to Transformers, recurrent and hybrid models, and long-context systems.
• Design and execute architecture-level experiments focused on scaling laws, memory mechanisms, training behavior, and compute-performance trade-offs.
• Prototype models end-to-end, translating research concepts into robust, training-ready implementations.
• Analyze model behavior, failure modes, inductive biases, and architectural strengths and limitations.
• Collaborate with inference and systems engineering teams to ensure new architectures are efficient, scalable, and suitable for deployment.
• Read, reproduce, evaluate, and extend cutting-edge machine learning research papers.
• Contribute to internal research notes, benchmarks, experiments, and open-source initiatives where applicable.
• Move fluidly between theoretical investigation, rapid experimentation, and production-oriented engineering.

Requirements:

• Strong foundation in machine learning and deep learning fundamentals, with practical experience applying them to model development.
• Hands-on experience implementing neural network or model architectures from scratch.
• Strong understanding of attention mechanisms, RNNs, state-space models, hybrid architectures, or related approaches.
• Solid knowledge of training dynamics, optimization, scaling behavior, and architecture-level performance considerations.
• Understanding of model-level memory, latency, compute, and efficiency constraints.
• Proficiency with PyTorch or JAX and the ability to develop and experiment with research-oriented ML code.
• Ability to evaluate architectural ideas through both theoretical reasoning and empirical experimentation.
• Strong communication skills, with the ability to clearly explain technical concepts and architectural trade-offs.
• Preferred experience with non-Transformer architectures such as RNN variants, state-space models, or long-context systems.
• Preferred background in research-driven startups, open-source machine learning projects, large-scale training, or custom training loops.
• Publications, preprints, notable research contributions, or experience with inference optimization and deployment constraints are advantageous.

Benefits:

• Competitive compensation and meaningful equity.
• Opportunity to work directly on core AI model architecture rather than focusing primarily on fine-tuning.
• Significant influence over technical and research direction within a rapidly growing organization.
• Small, high-caliber team with fast feedback loops and a strong research-oriented environment.
• Opportunity to take research concepts from experimentation through to production deployment.
• Full-time position with a globally distributed work environment.