– Amazon SageMaker HyperPod is a purpose-built infrastructure for optimizing foundation model (FM) training and inference at scale. SageMaker HyperPod removes the undifferentiated heavy lifting involved in building and optimizing machine learning (ML) infrastructure for training FMs, reducing training time by up to 40%.
SageMaker HyperPod offers persistent clusters with built-in resiliency, while also offering deep infrastructure control by allowing users to SSH into the underlying Amazon Elastic Compute Cloud (Amazon EC2) instances. It helps efficiently scale model development and deployment tasks such as training, fine-tuning, or inference across a cluster of hundreds or thousands of AI accelerators, while reducing the operational heavy lifting involved in managing such clusters. As AI moves towards deployment adopting to a multitude of domains and use cases, the need for flexibility and control is becoming more pertinent. Large enterprises want to make sure the GPU clusters follow the organization-wide policies and security rules. Mission-critical AI/ML workloads often require specialized environments that align with the organization’s software stack and operational standards.
SageMaker HyperPod supports Amazon Elastic Kubernetes Service (Amazon EKS) and offers two new features that enhance this control and flexibility to enable production deployment of large-scale ML workloads:
- Continuous provisioning – SageMaker HyperPod now supports continuous provisioning, which enhances cluster scalability through features like partial provisioning, rolling updates, concurrent scaling operations, and continuous retries when launching and configuring your HyperPod cluster.
- Custom AMIs – You can now use custom Amazon Machine Images (AMIs), which enables the preconfiguration of software stacks, security agents, and proprietary dependencies that would otherwise require complex post-launch bootstrapping. Customers can create custom AMIs using the HyperPod public AMI as a base and install additional software required to meet their organization’s specific security and compliance requirements.
In this post, we dive deeper into each of these features.
Continuous provisioning
The new continuous provisioning feature in SageMaker HyperPod represents a transformative advancement for organizations running intensive ML workloads, delivering unprecedented flexibility and operational efficiency that accelerates AI innovation. This feature provides the following benefits:
- Partial provisioning – SageMaker HyperPod prioritizes delivering the maximum possible number of instances without failure. You can start running your workload while your cluster will attempt to provision the remaining instances.
- Concurrent operations – SageMaker HyperPod supports simultaneous scaling and maintenance activities (such as scale up, scale down, and patching) on a single instance group waiting for previous operations to complete.
- Continuous retries – SageMaker HyperPod persistently attempts to fulfill the user’s request until it encounters a
NonRecoverableerror from where recovery is not possible. - Increased customer visibility – SageMaker HyperPod maps customer-initiated and service-initiated operations to structured activity streams.
Custom AMIs
With custom AMIs, you can align your ML environments with organizational security standards and software requirements. This allows for greater control over the environment and reduces the risk of compatibility issues.
By using custom AMIs, users can create a baseline image that includes all the necessary components for their ML workloads, such as drivers, libraries, and security agents. This simplifies the deployment process and ensures consistency across different environments. Furthermore, you can use custom AMIs to enforce organizational security policies and compliance requirements.
The ability to leverage existing infrastructure and pre-configured environments significantly reduces the time and effort required to get started with ML training and inference. This accelerates innovation and allows organizations to focus on building and deploying their models rather than managing infrastructure.
Source: Read the original article here.
Discover more tech insights on ByteTrending.
Discover more from ByteTrending
Subscribe to get the latest posts sent to your email.
