Make sure that any test that uses the network is properly marked

[leouieda]

We use a special network pytest mark to allow easily remove/select tests that require network access. From #331, it seems that not all our tests are properly marked as such.

It would be good to over the pooch/tests package and make sure that all test functions that access the internet are properly tagged (see

pooch/pooch/tests/test_core.py

Line 62 in 9c57dd3

@pytest.mark.network

for an example).

[jstorrs]

@tarunsamanta2k20 I don't know if what I am going to suggest is anywhere close to correct but this is what came to mind reading this:

First I would edit Makefile to duplicate the "test" target and add -m "not network" to the pytest call something like:

test-offline:
        # Run a tmp folder to make sure the tests are run on the installed version
        mkdir -p $(TESTDIR)
        cd $(TESTDIR); pytest $(PYTEST_ARGS) -m "not network" $(PROJECT)
        cp $(TESTDIR)/.coverage* .
        rm -r $(TESTDIR)

Based on https://docs.pytest.org/en/7.1.x/example/markers.html this should tell pytest to skip the tests that are labeled with "@pytest.mark.network".

After adding the new target to Makefile you should be able to run make test (full suite) or make test-offline (everything not marked network).

But I don't know how cache's affect things and whether pooch's tests already bypass cache. I suspect there's a way to point pooch at a temporary empty directory instead of the default cache. make test may already do that.

From there you can probably figure out which tests are using network by comparing runs of make test-offline while taking your computer on/offline and noticing changes in pass/fail status.

This is just my initial idea about how to go about it. At some point someone might think about whether there's an automated way to detect whether tests have been incorrectly marked as network.