PR: Add option to prepend or append Pythonpath Manager paths to sys.path #21769

mrclary · 2024-02-06T16:35:56Z

Description of Changes

Wrote at least one-line docstrings (for any new functions)
Added unit test(s) covering the changes (if testable)
Included a screenshot or animation (if affecting the UI, see Licecap)

UI/UX Changes

Added import button to update System Paths from the system PYTHONPATH environment variable.
Added toggle button that sets whether active paths should be prepended to or appended to sys.path in IPython console and Jedi completion environments.

Primary Changes

These changes directly affect the plugin API.

Spyder's transient configuration now includes user_paths, system_paths, and prioritize. user_paths and system_paths are dictionaries with path as key and enabled state as value. prioritize is a boolean. The unofficial configuration keys path, user_path, and system_path are no longer used. The spyder_pythonpath configuration key remains unchanged.
SPY_PYTHONPATH environment variable is eliminated in favor of directly accessing the spyder_pythonpath configuration key in the pythonpath_manager configuration section on startup of IPython console kernels.
sig_pythonpath_changed now sends the new spyder_pythonpath value (list of strings) and prioritize (boolean). This signal no longer sends the "old" paths. The old paths were only required by spyder-kernels in order to remove those paths from sys.path before adding the new paths to sys.path. The update to spyder-kernels in the companion PR eliminates the need for sending the "old" paths.
Fixed an issue where changes made by the user in the UI were applied even when the user presses the cancel button, if the system paths had changed on disk.
Fixed an issue where removing all user paths and then adding a user path resulted in an UnboundLocalError.

Secondary Changes

In order to simplify the handling of PYTHONPATH, the following changes were made to the Pythonpath Manager plugin. These do not directly affect the UI, UX, or plugin API, but streamline and clarify the actions performed within the plugin.

The internal attributes of the PythonpathContainer container object and the PathManager widget object were changed from path, not_active_path, user_path, system_path, and original_path_dict, to the less ambiguous user_paths and system_paths. The latter are maintained as OrderedDict objects throughout.
The pythonpath_manager configurations are now only set in PythonpathContainer._save_paths, and only on condition of change, rather than in multiple locations throughout the container and PathManager widget.
The sig_pythonpath_changed signal is emitted only in PythonpathContainer._save_paths, and only on condition that spyder_pythonpath or prioritize has changed.

Companion PRs

Outstanding questions

Should python-lsp-server.pylsp.Document.sys_path be removed (API break)?
What icon should be used for the prioritze button?

Issue(s) Resolved

Fixes #17066

Affirmation

By submitting this Pull Request or typing my (user)name below,
I affirm the Developer Certificate of Origin
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

I certify the above statement is true and correct:
@mrclary

pep8speaks · 2024-02-06T16:36:13Z

Hello @mrclary! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file spyder/plugins/pythonpath/container.py:

Line 170:1: W293 blank line contains whitespace

In the file spyder/plugins/pythonpath/widgets/pathmanager.py:

Line 503:80: E501 line too long (83 > 79 characters)
Line 506:80: E501 line too long (82 > 79 characters)

In the file spyder/utils/icon_manager.py:

Line 291:80: E501 line too long (101 > 79 characters)
Line 292:80: E501 line too long (102 > 79 characters)

Comment last updated at 2025-01-21 04:06:10 UTC

…pdate_paths method and call setup in update-paths method. This will allow the container to instantiate the PathManager widget before providing paths. Paths will not be retrieved or determined within the widget, only passed to it by the container.

These will be dictionaries and the container will handle updating the pythonpath_manager configuration and assembling the final spyder_pythonpath. There is no need for _update_system_path method because the container will handle updates to the underlying system path. Again, the widget will only handle user-interactive changes.

This will be done in the container instead.

…oritize) -> (_user_paths, _system_paths, _project_paths, _prioritize, _spyder_pythonpath). Path lists are now OrderedDict * Simplifies _load_pythonpath -> _load_paths * Move migration method from setup to _load_paths

* Promptly exits if remnants of old configuration are not present * Removes remnants of old configuration if present * Constructs user paths from old configuration remnants

* Configuration keys and private attributes for user paths, system paths, prioritize, and spyder_pythonpath are set conditionally in this method and nowhere else. * sig_pythonpath_changed is conditionally emitted from this method and nowhere else. This signal now sends only the spyder_pythonpath and prioritize, not the old spyder_pythonpath. Subscribers should update accordingly.

…ardly constructed from project, user, and system paths attributes.

…d in _save_paths if spyder_pythonpath is changed.

…d in PathManager.updat_paths

… Note that spyder-kernels must be updated to accommodate.

Icon and tooltip are changed to reflect current state.

Co-authored-by: Jitse Niesen <[email protected]> Typographical errors. Improved docstring clarity

The desired affect is project paths | user paths | system paths, where the paths are in that order and are overwritten in that order. System paths cannot overwrite user paths, which cannot overwrite project paths, i.e we cannot just do project_paths.update(user_paths) etc.

…path manager widget is invoked. If the system paths have changed since last widget invocation, then the user may not be aware and there is no indication in the widget that there has been a change. Furthermore, canceling the widget may still result in a change to the pythonpath, which would be inconsistent with the cancel action.

Rather than automatically updating the system paths, provide mechanism for user to do so.

* Test system PYTHONPATH import in test_pathmanager instead of test_mainwindow * Move restore_user_env fixture from app/tests/conftest.py to utils/tests/conftest.py * Ensure that the user environment script runs on posix while testing

…lary/spyder-kernels.git external-deps/spyder-kernels subrepo: subdir: "external-deps/spyder-kernels" merged: "66766d44b" upstream: origin: "https://github.com/mrclary/spyder-kernels.git" branch: "ppm-syspath" commit: "66766d44b" git-subrepo: version: "0.4.9" origin: "???" commit: "???"

ccordoba12

Thanks @mrclary, thanks for your work on this! This is an initial review for you.

spyder/plugins/completion/providers/languageserver/provider.py

spyder/plugins/ipythonconsole/widgets/shell.py

spyder/plugins/ipythonconsole/utils/tests/test_spyder_kernel.py

spyder/plugins/ipythonconsole/widgets/tests/test_kernelconnect.py

ccordoba12 · 2024-05-30T16:28:56Z

spyder/plugins/pythonpath/container.py

-    def on_close(self):
-        # Save current system path to detect changes next time Spyder starts
-        self.set_conf('system_path', get_system_pythonpath())


Why did you remove this?

This on_close would save the current system paths to the configuration on closing the plugin without user acknowledgement. With this PR, the configuration is saved and signal emitted only from PythonpathContainer._save_paths, after user interaction, to ensure that any changes are initiated and approved by the user. In this case, there is nothing that needs to be done when closing the container, so the function no longer is needed.

Please don't resolve this until I have a chance to test your PR manually.

spyder/plugins/pythonpath/widgets/pathmanager.py

ccordoba12 · 2025-01-19T17:55:39Z

spyder/plugins/pythonpath/widgets/tests/test_pathmanager.py

+    qtbot.waitUntil(pathmanager.prioritize_button.isChecked)
+    assert pathmanager.prioritize_button.isChecked()
+    assert pathmanager.button_ok.isEnabled()
+


I think a test for the new prioritize button is missing. Could you add one?

So the state of the OK button is tested with respect to changes to the prioritize button in test_buttons_state. The Pythonpath Manager doesn't actually do anything with the prioritize state except to pass its value via signal, i.e. it doesn't do any path sorting, so I'm not sure what should be tested here. spyder-kernels will actually do the sorting, and this is tested in test_console_kernel.py, added in the companion PR. Is there something in particular you had in mind to test?

Ok, but we need to check that the Spyder UI actually changes the priority of paths in the console at least (it's harder to check in the LSP).

For that then I suggest to extend test_PYTHONPATH_in_consoles, by clicking the prioritize button in the PPM dialog and then check in the console that the paths handled by it are at the front of sys.path.

Co-authored-by: Carlos Cordoba <[email protected]>

mrclary · 2025-01-21T18:24:34Z

@ccordoba12, thank you for reviewing this PR. I implemented most of your recommendations and have rebased onto the latest master locally, but haven't pushed them up just yet. I had a few follow-up comments which I can resolve after you get a chance to respond. After these are resolved, I'll push up all the changes for another round of review.

mrclary added type:Feature component:Pythonpath manager labels Feb 6, 2024

mrclary self-assigned this Feb 6, 2024

mrclary force-pushed the ppm-syspath branch 2 times, most recently from 3999697 to 3c2c64d Compare February 12, 2024 23:25

mrclary force-pushed the ppm-syspath branch 2 times, most recently from b11ccad to a433e97 Compare February 22, 2024 21:31

mrclary force-pushed the ppm-syspath branch 13 times, most recently from 5f68078 to ffd9395 Compare March 4, 2024 21:27

mrclary marked this pull request as ready for review March 4, 2024 22:11

mrclary marked this pull request as draft March 4, 2024 22:11

mrclary marked this pull request as ready for review March 4, 2024 22:14

mrclary requested review from ccordoba12, dalthviz and CAM-Gerlach March 4, 2024 22:14

mrclary force-pushed the ppm-syspath branch 3 times, most recently from 0a04027 to 73e407e Compare March 13, 2024 00:54

mrclary added 25 commits January 14, 2025 13:46

Remove superfluous user_path attribute

7230167

Remove algorithm to save system PYTHONPATH.

ddf2933

This will be done in the container instead.

Simplify get_user_paths and get_system_paths

ae3516d

Revise configuration migration method.

08d636d

* Promptly exits if remnants of old configuration are not present * Removes remnants of old configuration if present * Constructs user paths from old configuration remnants

Simplify get_spyder_pythonpath. spyder_pythonpath is now straightforw…

d37c785

…ardly constructed from project, user, and system paths attributes.

Simplify update_active_project_path. sig_pythonpath_changed is emitte…

1aaca0a

…d in _save_paths if spyder_pythonpath is changed.

Update show_path_manager method. Note that PathManager.setup is calle…

de445ba

…d in PathManager.updat_paths

Propagate changes to sig_pythonpath_changed to pythonpath plugin

0876d5f

Propagate changes to sig_pythonpath_changed to ipythonconsole plugin.…

7b22010

… Note that spyder-kernels must be updated to accommodate.

Propagate changes to sig_pythonpath_changed to completions plugin.

0f83e3e

Update main window test

8b4b4a2

Update export_pythonpath

348e0f7

Update widget icon.

7043dd7

Icon and tooltip are changed to reflect current state.

Apply suggestions from code review

c15876f

Co-authored-by: Jitse Niesen <[email protected]> Typographical errors. Improved docstring clarity

Apply suggestions from python-lsp-server code review

4c7724f

Do not emit sig_path_changed on closeEvent or reject.

a16c7dc

Add import path functionality.

67f3ccb

Rather than automatically updating the system paths, provide mechanism for user to do so.

Update unit tests

e13932f

* Test system PYTHONPATH import in test_pathmanager instead of test_mainwindow * Move restore_user_env fixture from app/tests/conftest.py to utils/tests/conftest.py * Ensure that the user environment script runs on posix while testing

mrclary force-pushed the ppm-syspath branch from b940e1d to 73415e6 Compare January 14, 2025 21:47

ccordoba12 reviewed Jan 19, 2025

View reviewed changes

mrclary and others added 2 commits January 20, 2025 09:26

Apply suggestions from code review

66a8f00

Co-authored-by: Carlos Cordoba <[email protected]>

Apply suggestions from code review

def6dae

Co-authored-by: Carlos Cordoba <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PR: Add option to prepend or append Pythonpath Manager paths to sys.path #21769

PR: Add option to prepend or append Pythonpath Manager paths to sys.path #21769

mrclary commented Feb 6, 2024 •

edited

Loading

pep8speaks commented Feb 6, 2024 •

edited

Loading

ccordoba12 left a comment

ccordoba12 May 30, 2024

mrclary Jan 20, 2025

ccordoba12 Jan 22, 2025

ccordoba12 Jan 19, 2025

mrclary Jan 21, 2025

ccordoba12 Jan 22, 2025

mrclary commented Jan 21, 2025

PR: Add option to prepend or append Pythonpath Manager paths to sys.path #21769

Are you sure you want to change the base?

PR: Add option to prepend or append Pythonpath Manager paths to sys.path #21769

Conversation

mrclary commented Feb 6, 2024 • edited Loading

Description of Changes

UI/UX Changes

Primary Changes

Secondary Changes

Companion PRs

Outstanding questions

Issue(s) Resolved

Affirmation

pep8speaks commented Feb 6, 2024 • edited Loading

Comment last updated at 2025-01-21 04:06:10 UTC

ccordoba12 left a comment

Choose a reason for hiding this comment

ccordoba12 May 30, 2024

Choose a reason for hiding this comment

mrclary Jan 20, 2025

Choose a reason for hiding this comment

ccordoba12 Jan 22, 2025

Choose a reason for hiding this comment

ccordoba12 Jan 19, 2025

Choose a reason for hiding this comment

mrclary Jan 21, 2025

Choose a reason for hiding this comment

ccordoba12 Jan 22, 2025

Choose a reason for hiding this comment

mrclary commented Jan 21, 2025

mrclary commented Feb 6, 2024 •

edited

Loading

pep8speaks commented Feb 6, 2024 •

edited

Loading