- A single Group can launch only same type of EC2 instance.
- A single Group can launch only one type of AMI
- AMI's need to be present in the same region, rather same zone
- We can not used this group to scale RDS instance
- If we keep ELB as one level higher than Auto Scaling Group, when Scale Down happens, the process instructs ELB to remove EC2 instance from ELB group first.
- Should be cautious about using Elastic IP with Auto Scaling Group. It may not be a cost effective option.
- Max # of Launch configuration per AWS account is 100.
Tensor Parallelism in GPU Tensor parallelism is a technique used to distribute the computation of large tensor operations across multiple GPUs or multiple cores within a GPU . It is an essential method for improving the performance and scalability of deep learning models, particularly when dealing with very large models that cannot fit into the memory of a single GPU. Key Concepts Tensor Operations : Tensors are multidimensional arrays used extensively in deep learning. Common tensor operations include matrix multiplication, convolution, and element-wise operations. Parallelism : Parallelism involves dividing a task into smaller sub-tasks that can be executed simultaneously. This approach leverages the parallel processing capabilities of GPUs to speed up computations. How Tensor Parallelism Works Splitting Tensors : The core idea of tensor parallelism is to split large tensors into smaller chunks that can be processed in parallel. Each chunk is assigned to a different GP...
Comments