Job Summary: We are seeking an experienced Runtime Software Architect to lead the design and development of an advanced runtime framework tailored for Neureality servers. This role involves analyzing existing runtime systems and machine learning runtime frameworks such as ONNX Runtime onnxruntime.ai, TensorFlow Serving, PyTorch's TorchServe, Apache TVM tvm.apache.org, and Glow github.com, to inform the creation of a new framework that efficiently leverages low-level hardware components, including DSPs, video decoders, ARM cores, and DLAs. The ideal candidate will possess a deep understanding of these technologies and a proven track record in architecting software that harmonizes with complex hardware infrastructures.

Key Responsibilities:

• Framework Analysis: Conduct comprehensive evaluations of existing runtime systems and ML runtime frameworks, such as ONNX Runtime, TensorFlow Serving, TorchServe, Apache TVM, VLLM, TRT-LLM, and Glow, to identify their strengths, weaknesses, and applicability to our objectives.
• Architecture Design: Develop a robust runtime framework optimized for Neureality servers, ensuring seamless integration with hardware components such as DSPs, video & audio decoders, ARM cores, and DLAs.
• Hardware Integration: Collaborate with hardware engineers to adapt and optimize software components, maximizing performance and efficiency across various hardware modules.
• Performance Optimization: Implement strategies to enhance system performance, including reducing latency, increasing throughput, and improving resource utilization.
• Cross-Functional Collaboration: Work closely with cross-functional teams to align software architecture with business goals and technological advancements.