Senior HPC Storage Architect
At Hudson River Trading (HRT) we are mathematicians, computer scientists, statisticians, physicists and engineers. We research and develop automated trading algorithms using advanced mathematical techniques. We have built one of the world's most sophisticated computing environments, and our researchers are at the forefront of innovation in the world of algorithmic trading.
Job Description
The Research & Development team at Hudson River Trading (HRT) builds and maintains the computers, networks, storage, operating systems, and software that allow our trading strategies and research environment to operate worldwide 24/7. HRT operates incredibly large GPU and CPU compute clusters, larger than most national labs. We have built or are in the process of building our own on-prem data centers globally and manage over 100 petabytes of storage to facilitate industry leading AI/ML based trading.
We are looking for a hands-on Storage Architect who enjoys being challenged, appreciates an open and collaborative organizational structure, and thrives in a fast-paced environment. In this role, you will own the vision, planning, and execution of HRT’s current and future storage needs. Given HRT’s growth and ever-evolving needs (especially when it comes to AI), this is a unique opportunity to set the direction for future storage solutions and work across engineering teams to deliver it. You will drive engagement with all stakeholders — internal users and both external and internal technology providers — understand their product portfolio and roadmap, and build a strategy for HRT that is inclusive of financial modeling.
Responsibilities
- Own the overall charter of delivering performant, reliable, and future-proof storage solutions for all of HRT’s storage needs
- Collaborate and drive efforts across multiple (internal) cross-functional engineering teams
- Improve user experience by understanding storage workloads and designing a variety of solutions and tools
- Forecast future storage needs, research new storage solutions, and build a five- and ten-year roadmap
- Draw insights from a deep understanding of hardware infrastructure and industry trends to inform the roadmap and planning decisions
- Review deliverables and provide relevant guidance to engineering teams
- Troubleshoot complex storage, OS, and networking issues
- Integrate storage systems with HPC clusters, ensuring compatibility with existing hardware and software
- Participate in an on-call rotation to ensure continuous support for HRT’s operations, responding promptly to critical storage issues and incidents.
Qualifications
- 10+ years of experience in HPC-like environments with multi-PB storage deployments
- Experience in developing advanced storage software solutions and/or leading software development team(s)
- Deep expertise in HPC and scale-out enterprise storage solutions
- Knowledge of distributed file systems used for large-scale cluster computing (Lustre, GPFS, WEKA, S3, CEPH, etc.)
- Strong leadership, communication, and stakeholder management skills
- Deep technical understanding of commodity storage technologies
- Knowledgeable on storage industry trends
- Deep technical understanding of server architecture, design, and development, with emphasis on power and performance analysis
- Ability to analyze and solve problems under quick turnaround times
- Ability to manage time efficiently, balancing independent and collaborative workflows
- Proficient in Python and UNIX/Linux shell scripting
The estimated base salary range for this position is $200,000 to $275,000 per year, based on job-related skills and experience. This role will also be eligible for discretionary performance-based bonuses and a competitive benefits package.
The Research & Development team at Hudson River Trading (HRT) builds and maintains the computers, networks, storage, operating systems, and software that allow our trading strategies and research environment to operate worldwide 24/7. HRT operates incredibly large GPU and CPU compute clusters, larger than most national labs. We have built or are in the process of building our own on-prem data centers globally and manage over 100 petabytes of storage to facilitate industry leading AI/ML based trading.
We are looking for a hands-on Storage Architect who enjoys being challenged, appreciates an open and collaborative organizational structure, and thrives in a fast-paced environment. In this role, you will own the vision, planning, and execution of HRT’s current and future storage needs. Given HRT’s growth and ever-evolving needs (especially when it comes to AI), this is a unique opportunity to set the direction for future storage solutions and work across engineering teams to deliver it. You will drive engagement with all stakeholders — internal users and both external and internal technology providers — understand their product portfolio and roadmap, and build a strategy for HRT that is inclusive of financial modeling.
Responsibilities
- Own the overall charter of delivering performant, reliable, and future-proof storage solutions for all of HRT’s storage needs
- Collaborate and drive efforts across multiple (internal) cross-functional engineering teams
- Improve user experience by understanding storage workloads and designing a variety of solutions and tools
- Forecast future storage needs, research new storage solutions, and build a five- and ten-year roadmap
- Draw insights from a deep understanding of hardware infrastructure and industry trends to inform the roadmap and planning decisions
- Review deliverables and provide relevant guidance to engineering teams
- Troubleshoot complex storage, OS, and networking issues
- Integrate storage systems with HPC clusters, ensuring compatibility with existing hardware and software
- Participate in an on-call rotation to ensure continuous support for HRT’s operations, responding promptly to critical storage issues and incidents.
Qualifications
- 10+ years of experience in HPC-like environments with multi-PB storage deployments
- Experience in developing advanced storage software solutions and/or leading software development team(s)
- Deep expertise in HPC and scale-out enterprise storage solutions
- Knowledge of distributed file systems used for large-scale cluster computing (Lustre, GPFS, WEKA, S3, CEPH, etc.)
- Strong leadership, communication, and stakeholder management skills
- Deep technical understanding of commodity storage technologies
- Knowledgeable on storage industry trends
- Deep technical understanding of server architecture, design, and development, with emphasis on power and performance analysis
- Ability to analyze and solve problems under quick turnaround times
- Ability to manage time efficiently, balancing independent and collaborative workflows
- Proficient in Python and UNIX/Linux shell scripting
The estimated base salary range for this position is $200,000 to $275,000 per year, based on job-related skills and experience. This role will also be eligible for discretionary performance-based bonuses and a competitive benefits package.
