Use Nsight System to Profile a Model Training with DeepSpeed on Multi-Node Cluster
This post is to log how I managed to profile a model training running on multiple nodes in a cluster with DeepSpeed and Nsight System. Click here to jump to ...
Recent Posts
Profiling of different implementation of attention modules
Placeholder for the blog logging the results of profiling vllm
Profiling of different implementation of attention modules
Placeholder for the blog logging the results of profiling different implementations of attention modules.
Build an App with OpenAI API
Placeholder for the blog logging how I build an app with OpenAI API.
Training Custom Mixtral Model with DeepSpeed
Placeholder for the blog logging how I trained a custom Mixtral model with DeepSpeed.