Please ignore secret bonuses. Secret tests do NOT award bonus. Max hw grade is 30+2 bonus efficiency

Do you need help?

Line Profilers for Gentlemen in a Hurry

S
Silktrader (2550 points)
2 6 16
in Programming in Python by (2.6k points)

The first homework has already, worryingly, set a trend to (needlessly) micro-optimise the simplest of statements. In a bid to dissect and extinguish the lamest parts of my scripts I've resorted to using a line profiler, which describes the number of calls and execution time of every line of code. The common cProfiler instead presents aggregate information about function calls.

The "Software and Commands" page mentions a line profiler that works within Spyder, although I had multiple issues with it. For starters I was prompted to update or install 239**34 other required packages, change my family name and convert to Buddhism. It doesn't seem to be possible to run Spyder's line profiler outside of the IDE (Shift-F10) and without selecting each script to be profiled from the GUI. Now, I wouldn't touch Anaconda's stock IDE with a barge pole, so I find the command line based kernprof line profiler more convenient.

The steps to set it up were the following:

  • conda install -c anaconda line_profiler in "Anaconda's Prompt"

  • decorate the functions to be profiled with @profile

  • run kernprof -l -v script_name.py

One can also run "pytest" files with "kernprof", which will analyse all the decorated functions called by the tests. Unfortunately, running test files on their own, without relying on "kernprof" and while the decorator's enabled, throws an error as "pytest" complains that the "profile" module can't be found. I comment out the offending line when needed.

This is the kind of result one can expect. The line above the function's definition includes the decorator:

My gripes with the line profiler are two:

  1. it doesn't allow one to run multiple tests and average results (as with timeit, for instance) — one must manually set up multiple calls to decorated functions and look at aggregates
  2. pytest and both line profilers don't play well together whilst pytest's own profiler doesn't output execution statistics line by line

It'd be nice if anybody came up with ways to address either point.

724 views
closed

2 Answers

Best answer
andrea.sterbini (207920 points)
750 1267 2373
by (208k points)
selected by
See my post on the Italian forum for a solution to leave @profile when doing pytest
S
Silktrader (2550 points)
2 6 16
by (2.6k points)

I assume that this is the answer you're referring to. You suggested to define a @profile decorator at the start of the module when the line profiler doesn't inject its own:

if 'profile' not in __builtins__:
    def profile(x): return x 

Works. Thanks!

Additionally, this comment will prevent PyCharm from complaining that @profile is potentially undefined:

# noinspection PyUnboundLocalVariable
angelo.spognardi (8170 points)
75 155 224
by (8.2k points)
S
Silktrader (2550 points)
2 6 16
by (2.6k points)

I did follow the guide when I first set up my environment. I linked to it in the post above.

Thanks.