Accelerate documentation
Logging with Accelerate
Getting started
Tutorials
OverviewMigrating to 🤗 AccelerateLaunching distributed codeLaunching distributed training from Jupyter NotebooksTroubleshooting guide
How-To Guides
Start Here!Example ZooHow to perform inference on large models with small resourcesKnowing how big of a model you can fit into memoryHow to quantize modelHow to perform distributed inference with normal resourcesPerforming gradient accumulationAccelerating training with local SGDSaving and loading training statesUsing experiment trackersHow to use Apple Silicon M1 GPUsHow to train in low precision (FP8)How to use DeepSpeedHow to use Fully Sharded Data ParallelismHow to use Megatron-LMHow to use 🤗 Accelerate with SageMakerHow to use 🤗 Accelerate with Intel® Extension for PyTorch for cpu
Concepts and fundamentals
🤗 Accelerate's internal mechanismLoading big models into memoryComparing performance across distributed setupsExecuting and deferring jobsGradient synchronizationHow training in low-precision environments is possible (FP8)TPU best practices
Reference
Main Accelerator classStateful configuration classesThe Command LineTorch wrapper classesExperiment trackersDistributed launchersDeepSpeed utilitiesLoggingWorking with large modelsDistributed inference with big modelsKwargs handlersUtility functions and classesMegatron-LM UtilitiesFully Sharded Data Parallelism Utilities
You are viewing v0.27.2 version. A newer version v1.14.0 is available.
Logging with Accelerate
Refer to the Troubleshooting guide or to the example below to learn how to use 🤗 Accelerate’s logger.
accelerate.logging.get_logger
< source >( name: str log_level: str = None )
Returns a logging.Logger for name that can handle multiprocessing.
If a log should be called on all processes, pass main_process_only=False If a log should be called on all
processes and in order, also pass in_order=True
Example:
>>> from accelerate.logging import get_logger
>>> from accelerate import Accelerator
>>> logger = get_logger(__name__)
>>> accelerator = Accelerator()
>>> logger.info("My log", main_process_only=False)
>>> logger.debug("My log", main_process_only=True)
>>> logger = get_logger(__name__, log_level="DEBUG")
>>> logger.info("My log")
>>> logger.debug("My second log")
>>> array = ["a", "b", "c", "d"]
>>> letter_at_rank = array[accelerator.process_index]
>>> logger.info(letter_at_rank, in_order=True)