Can I restrict nose coverage output to directory (rather than package)?

Question

My SUT looks like:

foo.py
bar.py
tests/__init__.py [empty]
tests/foo_tests.py
tests/bar_tests.py
tests/integration/__init__.py [empty]
tests/integration/foo_tests.py
tests/integration/bar_tests.py

When I run nosetests --with-coverage, I get details for all sorts of modules that I'd rather ignore. But I can't use the --cover-package=PACKAGE option because foo.py & bar.py are not in a package. (See the thread after http://lists.idyll.org/pipermail/testing-in-python/2008-November/001091.html for my reasons for not putting them in a package.)

Can I restrict coverage output to just foo.py & bar.py?

Update - Assuming that there isn't a better answer than Nadia's below, I've asked a follow up question: "How do I write some (bash) shell script to convert all matching filenames in directory to command-line options?"

Solution 1

You can use it like this:

--cover-package=foo --cover-package=bar

I had a quick look at nose source code to confirm: This is the line

    if options.cover_packages:
        for pkgs in [tolist(x) for x in options.cover_packages]:

Solution 2

You can use:

--cover-package=.

or even set environment variable

NOSE_COVER_PACKAGE=.

Tested with nose 1.1.2

Solution 3

I have a lot of top-level Python files/packages and find it annoying to list them all manually using --cover-package, so I made two aliases for myself. Alias nosetests_cover will run coverage with all your top-level Python files/packages listed in --cover-package. Alias nosetests_cover_sort will do the same and additionally sort your results by coverage percentage.

nosetests_cover_cmd="nosetests --with-coverage --cover-erase --cover-inclusive --cover-package=\$( ls | sed -r 's/[.]py$//' | fgrep -v '.' | paste -s -d ',' )"
alias nosetests_cover=$nosetests_cover_cmd
alias nosetests_cover_sort="$nosetests_cover_cmd 2>&1 | fgrep '%' | sort -nr -k 4"

Notes:

• This is from my .bashrc file. Modify appropriately if you don't use bash.
• These must be run from your top-level directory. Otherwise, the package names will be incorrect and coverage will silently fail to process them (i.e. instead of telling you your --cover-package is incorrect, it will act like you didn't supply the option at all).
• I'm currently using Python 2.7.6 on Ubuntu 13.10, with nose version 1.3.0 and coverage version 3.7.1. This is the only setup in which I've tested these commands.
• In your usage, remove --cover-erase and --cover-inclusive if they don't match your needs.
• If you want to sort in normal order instead of reverse order, replace -nr with -n in the sort command.
• These commands assume that all of your top-level Python files/packages are named without a dot (other than the dot in ".py"). If this is not true for you, read Details section below to understand the command parts, then modify the commands as appropriate.

Details:

I don't claim that these are the most efficient commands to achieve the results I want. They're just the commands I happened to come up with. =P

The main thing to explain would be the argument to --cover-package. It builds the comma-separated list of top-level Python file/package names (with ".py" stripped from file names) as follows:

• \$ -- Escapes the $ character in a double-quoted string.
• $( ) -- Inserts the result of the command contained within.
• ls -- Lists all names in current directory (must be top-level Python directory).
• | sed -r 's/[.]py$//' -- In the list, replaces "foo_bar.py" with "foo_bar".
• | fgrep -v '.' -- In the list, removes all names without a dot (e.g. removes foo_bar.pyc and notes.txt).
• | paste -s -d ',' -- Changes the list from newline-separated to comma-separated.

I should also explain the sorting.

• 2>&1 -- Joins stderr and stdout.
• | fgrep '%' -- Removes all output lines without a % character.
• | sort -nr -k 4 -- Sorts the remaining lines in reverse numerical order by the 4th column (which is the column for coverage percentage). If you want normal order instead of reverse order, replace -nr with -n.

Hope this helps someone! =)

Solution 4

If you use coverage:py 3.0, then code in the Python directory is ignored by default, including the standard library and all installed packages.

Solution 5

I would do this:

nosetests --with-coverage --cover-package=foo,bar tests/*

I prefer this solution to the others suggested; it's simple yet you are explicit about which packages you wish to have coverage for. Nadia's answer involves a lot more redundant typing, Stuart's answer uses sed and still creates a package by invoking touch __init__.py, and --cover-package=. doesn't work for me.

View more solutions

Comments

[Daryl Spitzer]

I'm impressed that you looked at the source. (I should have thought of that.) But it shows I need to use "--cover-package=foo --cover-package=bar". If you modify your answer I'll accept it (and vote it up).

[Nadia Alramli]

Thanks for the note. I fixed the answer. I've never used nose before. But I thought I'd give the question a try since no one else answered.

[Nadia Alramli]

Opps, sorry about the mistakes. Fixed :)

[merwok]

How can one achieve the same thing with options in setup.cfg?

[Skylar Saveland]

I get coverage reported from site-packages using this :/

[Michel Samia]

Do you use virualenv? I get also this, in virtualenv

[user3776598]

Any idea how to handle in a virtualenv?

[Ehsan Kia]

I'm sorry, but this is just wrong. That line you found doesn't prove anything, it just shows that the flag is converted and passed to the coverage tool as the "source=" keyword argument.The coverage tool takes paths as shown here: coverage.readthedocs.io/en/v4.5.x/source.html