scalene python cpu&gpu 性能分析工具

scalene 使用一个python cpu&gpu 性能分析工具，同时也支持内存的分析，同时还提供了基于ai的智能优化推荐

包含的一些特性

• cli 支持多种输出格式
• 包含了一个web-gui
• 基于ai 的智能提示

参考使用

• 安装

pip install scalene

• 包含的cli

usage: scalene [-h] [--version] [--column-width COLUMN_WIDTH] [--outfile OUTFILE] [--html] [--json] [--cli] [--stacks] [--web] [--no-browser] [--viewer] [--reduced-profile]

               [--profile-interval PROFILE_INTERVAL] [--cpu] [--cpu-only] [--gpu] [--memory] [--profile-all] [--profile-only PROFILE_ONLY] [--profile-exclude PROFILE_EXCLUDE]

               [--use-virtual-time] [--cpu-percent-threshold CPU_PERCENT_THRESHOLD] [--cpu-sampling-rate CPU_SAMPLING_RATE] [--allocation-sampling-window ALLOCATION_SAMPLING_WINDOW]

               [--malloc-threshold MALLOC_THRESHOLD] [--program-path PROGRAM_PATH] [--memory-leak-detector] [--on | --off]

 

Scalene: a high-precision CPU and memory profiler, version 1.5.36 (2024.03.04)

https://github.com/plasma-umass/scalene

 

command-line:

  % scalene [options] your_program.py [--- --your_program_args] 

or

  % python3 -m scalene [options] your_program.py [--- --your_program_args] 

 

in Jupyter, line mode:

  %scrun [options] statement

 

in Jupyter, cell mode:

  %%scalene [options]

   your code here

 

 

options:

  -h, --help            show this help message and exit

  --version             prints the version number for this release of Scalene and exits

  --column-width COLUMN_WIDTH

                        Column width for profile output (default: 132)

  --outfile OUTFILE     file to hold profiler output (default: stdout)

  --html                output as HTML (default: web)

  --json                output as JSON (default: web)

  --cli                 forces use of the command-line

  --stacks              collect stack traces

  --web                 opens a web tab to view the profile (saved as 'profile.html')

  --no-browser          doesn't open a web tab; just saves the profile ('profile.html')

  --viewer              opens the Scalene web UI.

  --reduced-profile     generate a reduced profile, with non-zero lines only (default: False)

  --profile-interval PROFILE_INTERVAL

                        output profiles every so many seconds (default: inf)

  --cpu                 profile CPU time (default:  True )

  --cpu-only            profile CPU time (deprecated: use --cpu )

  --gpu                 profile GPU time and memory (default: False )

  --memory              profile memory (default: True )

  --profile-all         profile all executed code, not just the target program (default: only the target program)

  --profile-only PROFILE_ONLY

                        profile only code in filenames that contain the given strings, separated by commas (default: no restrictions)

  --profile-exclude PROFILE_EXCLUDE

                        do not profile code in filenames that contain the given strings, separated by commas (default: no restrictions)

  --use-virtual-time    measure only CPU time, not time spent in I/O or blocking (default: False)

  --cpu-percent-threshold CPU_PERCENT_THRESHOLD

                        only report profiles with at least this percent of CPU time (default: 1%)

  --cpu-sampling-rate CPU_SAMPLING_RATE

                        CPU sampling rate (default: every 0.01s)

  --allocation-sampling-window ALLOCATION_SAMPLING_WINDOW

                        Allocation sampling window size, in bytes (default: 10485767 bytes)

  --malloc-threshold MALLOC_THRESHOLD

                        only report profiles with at least this many allocations (default: 100)

  --program-path PROGRAM_PATH

                        The directory containing the code to profile (default: the path to the profiled program)

  --memory-leak-detector

                        EXPERIMENTAL: report likely memory leaks (default: True)

  --on                  start with profiling on (default)

  --off                 start with profiling off

 

When running Scalene in the background, you can suspend/resume profiling

for the process ID that Scalene reports. For example:

 

   % python3 -m scalene  yourprogram.py &

 Scalene now profiling process 12345

   to suspend profiling: python3 -m scalene.profile --off --pid 12345

   to resume profiling:  python3 -m scalene.profile --on  --pid 12345

 

(venv) (venv) (base) ➜  dremio-sqlalchemy scalene app.py 

297511 E8:CB:ED:5A:52:41

(venv) (venv) (base) ➜  dremio-sqlalchemy scalene --help

usage: scalene [-h] [--version] [--column-width COLUMN_WIDTH] [--outfile OUTFILE] [--html] [--json] [--cli] [--stacks] [--web] [--no-browser] [--viewer] [--reduced-profile]

               [--profile-interval PROFILE_INTERVAL] [--cpu] [--cpu-only] [--gpu] [--memory] [--profile-all] [--profile-only PROFILE_ONLY] [--profile-exclude PROFILE_EXCLUDE]

               [--use-virtual-time] [--cpu-percent-threshold CPU_PERCENT_THRESHOLD] [--cpu-sampling-rate CPU_SAMPLING_RATE] [--allocation-sampling-window ALLOCATION_SAMPLING_WINDOW]

               [--malloc-threshold MALLOC_THRESHOLD] [--program-path PROGRAM_PATH] [--memory-leak-detector] [--on | --off]

 

Scalene: a high-precision CPU and memory profiler, version 1.5.36 (2024.03.04)

https://github.com/plasma-umass/scalene

 

command-line:

  % scalene [options] your_program.py [--- --your_program_args] 

or

  % python3 -m scalene [options] your_program.py [--- --your_program_args] 

 

in Jupyter, line mode:

  %scrun [options] statement

 

in Jupyter, cell mode:

  %%scalene [options]

   your code here

 

 

options:

  -h, --help            show this help message and exit

  --version             prints the version number for this release of Scalene and exits

  --column-width COLUMN_WIDTH

                        Column width for profile output (default: 132)

  --outfile OUTFILE     file to hold profiler output (default: stdout)

  --html                output as HTML (default: web)

  --json                output as JSON (default: web)

  --cli                 forces use of the command-line

  --stacks              collect stack traces

  --web                 opens a web tab to view the profile (saved as 'profile.html')

  --no-browser          doesn't open a web tab; just saves the profile ('profile.html')

  --viewer              opens the Scalene web UI.

  --reduced-profile     generate a reduced profile, with non-zero lines only (default: False)

  --profile-interval PROFILE_INTERVAL

                        output profiles every so many seconds (default: inf)

  --cpu                 profile CPU time (default:  True )

  --cpu-only            profile CPU time (deprecated: use --cpu )

  --gpu                 profile GPU time and memory (default: False )

  --memory              profile memory (default: True )

  --profile-all         profile all executed code, not just the target program (default: only the target program)

  --profile-only PROFILE_ONLY

                        profile only code in filenames that contain the given strings, separated by commas (default: no restrictions)

  --profile-exclude PROFILE_EXCLUDE

                        do not profile code in filenames that contain the given strings, separated by commas (default: no restrictions)

  --use-virtual-time    measure only CPU time, not time spent in I/O or blocking (default: False)

  --cpu-percent-threshold CPU_PERCENT_THRESHOLD

                        only report profiles with at least this percent of CPU time (default: 1%)

  --cpu-sampling-rate CPU_SAMPLING_RATE

                        CPU sampling rate (default: every 0.01s)

  --allocation-sampling-window ALLOCATION_SAMPLING_WINDOW

                        Allocation sampling window size, in bytes (default: 10485767 bytes)

  --malloc-threshold MALLOC_THRESHOLD

                        only report profiles with at least this many allocations (default: 100)

  --program-path PROGRAM_PATH

                        The directory containing the code to profile (default: the path to the profiled program)

  --memory-leak-detector

                        EXPERIMENTAL: report likely memory leaks (default: True)

  --on                  start with profiling on (default)

  --off                 start with profiling off

 

When running Scalene in the background, you can suspend/resume profiling

for the process ID that Scalene reports. For example:

 

   % python3 -m scalene  yourprogram.py &

 Scalene now profiling process 12345

   to suspend profiling: python3 -m scalene.profile --off --pid 12345

   to resume profiling:  python3 -m scalene.profile --on  --pid 12345

• 使用

scalene app.py

• 效果

说明

scalene 同时还有一个与比较工具的比较，scalene 支持的功能还是很多的，很值得试用下，尤其是ai 开发的场景

参考资料

https://github.com/plasma-umass/scalene