ML/AI Software Engineer – Optimization for Training, Fine‑Tuning, RL

WHAT YOU DO AT AMD CHANGES EVERYTHING
At AMD, our mission is to build great products that accelerate next-generation computing experiences—from AI and data centers, to PCs, gaming and embedded systems. Grounded in a culture of innovation and collaboration, we believe real progress comes from bold ideas, human ingenuity and a shared passion to create something extraordinary. When you join AMD, you'll discover the real differentiator is our culture. We push the limits of innovation to solve the world's most important challenges—striving for execution excellence, while being direct, humble, collaborative, and inclusive of diverse perspectives. Join us as we shape the future of AI and beyond.
Together, we advance your career.
The Person
You have strong technical and analytical skills, with experience building and operating end‑to‑end ML pipelines for training, fine‑tuning, reinforcement learning, and agentic systems. You are a capable problem solver and a technical leader who can define goals and scope, then drive development efforts. You possess strong communication skills, are enthusiastic about AI technologies, and are highly motivated to enable our customers with feature‑rich, efficient solutions.

The Role
We are looking for an experienced ML/AI software engineer with deep expertise in high‑level training, fine‑tuning, RLHF/RLVR, and agent frameworks for LLMs and MoE models. You will focus on software optimization to enable scalable fine‑tuning and RL pipelines; support robust agentic capabilities; integrate with leading open‑source ecosystems; and drive end‑to‑end performance on the AMD ROCm platform across AMD Radeon and Ryzen product families.

Key Responsibilities

• Support and optimize ROCm software for open‑source libraries (e.g., Hugging Face Transformers/PEFT/TRL, vLLM or similar serving stacks).
• Collaborate with kernel/compiler/platform teams to surface performance bottlenecks and capitalize on ROCm libraries and AMD Radeon/Ryzen hardware features.
• With proper understanding of leading-edge fine‑tuning pipelines for LLMs, MoE and multimodal models, including SFT, PEFT/LoRA/QLoRA, and mixed‑precision workflows, design, implement, and optimize ROCm software for such use cases.
• Enable and optimize RLHF/RLVR pipelines on AMD ROCm platform with support of popular policy optimization (PPO/DPO/ORPO/GRPO) algorithms.
• Efficient support State-of-the-Art agent frameworks and tool‑use capabilities on AMD ROC platform.
• Track and contribute to upstream ecosystems; propose enhancements aligned with our workloads and customer requirements.

Preferred Experience

• Strong C/C++ and Python development experience.
• Hands-on experience and knowledge on AI model training, finetuning, and inferencing using popular frameworks, as well as their performance optimization.
• Knowledge and deep understanding of State-of-the-Art Deep Learning and Deep Reinforcement Learning technologies and algorithms.
• Performance tuning on AMD GPUs and familiarity with the ROCm software stack (HIP runtime, rocBLAS, RCCL, MIOpen) is plus
• Contributions to open‑source projects in ML fine‑tuning, RL, or agent ecosystems are a plus.

Academic Credentials

• Bachelor's, Master's, or PhD in Computer Science, Electrical Engineering, or a related field.

LOCATION:
Nis or Belgrade (Serbia)

Benefits offered are described:
AMD benefits at a glance.

AMD does not accept unsolicited resumes from headhunters, recruitment agencies, or fee-based recruitment services. AMD and its subsidiaries are equal opportunity, inclusive employers and will consider all applicants without regard to age, ancestry, color, marital status, medical condition, mental or physical disability, national origin, race, religion, political and/or third-party affiliation, sex, pregnancy, sexual orientation, gender identity, military or veteran status, or any other characteristic protected by law. We encourage applications from all qualified candidates and will accommodate applicants' needs under the respective laws throughout all stages of the recruitment and selection process.