Interactive Python: cannot get `%lprun` to work, although line_profiler is imported properly

28,840

Solution 1

You have two ways to make work %lprun, one solution is temporal, i.e., it lasts until you finish your ipython session, and the other one is permanent.

Temporal: (as Carlos Cordoba's answer)

After starting ipython run the following:

In [1]: %load_ext line_profiler

Permanent:

Add the following lines to ~/.ipython/profile_default/ipython_config.py:

c.TerminalIPythonApp.extensions = [
    'line_profiler',
]

If you don't have the file ~/.ipython/profile_default/ipython_config.py, you can create it by (see this for more info):

ipython profile create

Solution 2

To make %lprun work, you need to load the extension into your session, using this command:

In [1]: %load_ext line_profiler

Check out this notebook to see some examples that use the magic.

Besides, if you are working with Spyder, there is also a third-party line_profiler plugin, which you can find here.

Share:
28,840
Mike Williamson
Author by

Mike Williamson

Slowly moving away from doing the fun technical work. :(

Updated on July 09, 2022

Comments

  • Mike Williamson
    Mike Williamson almost 2 years

    Problem

    Most iPython "magic functions" work fine for me right off the bat: %hist, %time, %prun, etc. However, I noticed that %lprun could not be found with iPython as I'd installed it originally.

    Attempt to Resolve

    I then discovered that I should install the line_profiler module. I have installed this module, but still cannot seem to get the magic function to work correctly. If I attempt to call %lprun, iPython still cannot find the function. If I call it with the full name ( line_profiler.magic_lprun ), the function can be found, but I cannot get it to work at all. Below is an example of what I'd done (which is taken step by step from "Python for Data Analysis" book):

    Success Using %prun

    [In:]

    def add_and_sum(x, y):
        added = x + y
        summed = added.sum(axis=1)
        return summed
    
    x = randn(3000, 3000)
    y = randn(3000, 3000)
    
    add_and_sum(x, y)
    

    With this I get a nice answer, as expected:

    [Out:]

    array([-23.6223074 , -10.08590736, -31.2957222 , ..., -14.17271747,
        63.84057725, -50.28469621])
    

    And I can do the profiling magic function %prun:

    [In:]

    %prun add_and_sum(x, y)
    

    [Out:]

    6 function calls in 0.042 seconds
    
    Ordered by: internal time
    
    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.020    0.020    0.029    0.029 <ipython-input-27-19f64f63ba0a>:1(add_and_sum)
        1    0.013    0.013    0.042    0.042 <string>:1(<module>)
        1    0.009    0.009    0.009    0.009 {method 'reduce' of 'numpy.ufunc' objects}
        1    0.000    0.000    0.009    0.009 _methods.py:16(_sum)
        1    0.000    0.000    0.009    0.009 {method 'sum' of 'numpy.ndarray' objects}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
    

    Fail Using %lprun

    But when I try %lprun, I cannot get anything:

    [In:]

    %lprun -f add_and_sum add_and_sum(x, y)
    

    [Out:]

    ERROR: Line magic function `%lprun` not found.
    

    And if I try to call the function with its standard name, it also does not work:

    [In:]

    line_profiler.magic_lprun -f add_and_sum.test test.add_and_sum(x, y)
    

    [Out:]

    line_profiler.magic_lprun -f add_and_sum.test test.add_and_sum(x, y)
                                           ^
    SyntaxError: invalid syntax
    

    But the library has been imported properly, or at least this is what it says:

    [In:]

    line_profiler
    

    [Out:]

    <module 'line_profiler' from '/Users/<edit>/anaconda/lib/python2.7/site-packages/line_profiler-1.0b3-py2.7-macosx-10.5-x86_64.egg/line_profiler.pyc'>
    

    [In:]

    line_profiler.magic_lprun
    

    [Out:]

    <function line_profiler.magic_lprun>
    

    It seems as if there is something extra that I am supposed to configure so that these new magic functions that I add can be identified as such. I could not find anything via a web search.

    I am running Spyder as an IDE (still using iPython as the console), but I have also tried it directly with iPython and with iPython notebook. I have had no luck in any format.

  • Mike Williamson
    Mike Williamson over 10 years
    Awesome, thanks so much, Carlos!! This worked for me... I could not seem to find this information anywhere on the web.
  • cd98
    cd98 almost 10 years
    +1 for the line_profiler plugin if you're using Spyder. I find it's the least troublesome way to run line_profiler and the ouput it gives is nice.
  • Alleo
    Alleo almost 8 years
    sorry to say, but it doesn't work anymore (since IPython 5.)
  • Carlos Cordoba
    Carlos Cordoba almost 8 years
    @Alleo, not my fault :-) The line_profiler extension hasn't been updated to work with IPython 5.0. Please see github.com/rkern/line_profiler/pull/65 for a fix
  • Steven C. Howell
    Steven C. Howell over 7 years
    %load_ext line_profiler command works in IPython 5.0 with line_profiler version 2.0.