https://github.com/spyder-ide/spyder/issues/1415
Spyder user config ambiguities with multiple Python environments #1415
3 participants
spyder-issues-migrator added this to the v2.4 milestone on 18 Feb 2015spyder-issues-migrator added imported 2–5 stars other Settings labels on 18 Feb 2015
From ccordoba12 on 2013-07-09T13:20:39Z
Jed, I've been thinking a lot on this problem for the last couple of weeks, and these are my thoughts:
- About the interpreters issue:
I think the best approach is to create a new "Interpreter" entry in "Preferences" in the same spirit as the one "Python Tools for Visual Studio" (PTVS) has (I'm attaching an image of it, please take a look at it before reading what's next).
I know you are opposed to the idea, but IMHO this will have several advantages:
a) The default interpreter will always be the one in which Spyder is running (i.e. sys.executable). I think this would be very intuitive for people: if you start Spyder with Anaconda's python, you'd expect that your consoles be Anaconda ones. The same with PythonXY and virtualenvs.
With this simple change (I think) beginners and causal users wouldn't even need to bother to go to this entry and set things by hand. Besides, it would make very easy to switch between distributions.
b) The user could add more interpreters and we'd be able to save their paths and names very easily, so that he could switch between them at will. Now that Anaconda is making so easy to create scientific python envs, I think people will start to use them more frequently and we need to provide a good alternative to handle them.
c) We could add a per interpreter configuration pane, as a dialog attached to a button on the Interpreter's entry.
We could move to it the "Advanced settings" and "External modules" tabs of the "Console" page, and a new tab to the let the user add the modules he is not having code completion on. This is necessary because every module with C/C++ extensions can't be introspected correctly by rope (see issue #1469 ).
d) We could add a dedicated entry in our spyder.ini to save all these options for every interpreter (in the same way as the "Run Settings" dialog saves run options for every file).
- About the clash between Python 2 and 3:
a) Given that (as you mentioned) configparser is quite different between 2 and 3, I should say we need a different ini file for each one.
This could also make our life much easier while testing Spyder, because we wouldn't need to reset spyder.ini before switching from 2 to 3.
b) We have to forbid using a Python 2 interpreter while Spyder is running under Python 3 and viceversa. This leads to ugly crashes and full window freezes if one tries to use IPython.
The only thing we should allow is switching to different interpreters of the same Python version.
I would like to revert the changes I introduced in issue #1365 and then start working on this in my personal clone to include it in 2.3. This way we could discuss about it without messing the main tree.
What do you think?
Attachment: ptvs-interpreter.png
From ccordoba12 on 2013-07-13T08:06:35Z
Pierre, I really would like to hear your opinion on this one. I think we need to solve it for 2.3 (although Jed disagrees with me) because correctly handling different interpreters is becoming a priority, not only for Mac and Anaconda users but also maintain sane defaults if people wants to use 2.7 and 3.3 at the same time.
Labels: MS-v2.3
From jed.lud...@gmail.com on 2013-07-13T09:24:28Z
To be clear, I completely agree that we need to solve this ambiguity quickly. I'm just not convinced that we need to add the complexity of configuring various interpreters.
After some discussion surrounding this topic on the mailing list [1], I'm more convinced than ever that we either have to buckle down and go all the way to something as rich as PyDev, i.e., multiple configurable environments assignable per-project which are used for all code completion and introspection, or we have to remain as simple as possible to avoid confusing people, especially new users (see issue #1402 ). Since our development resources are few and PyDev already exists, I'm very much in favor of remaining simple.
To that end, I threw together a quick prototype of a possible solution. It's not completely polished. It isn't fully Python 3 compatible yet. But it serves to illustrate the design approach. I wanted to present the idea before investing all the time required to get it fully ready. I created a bookmark called jedludlow-issue-1415 in my clone [2] that has a series of five commits beginning at revision c6e84f90c5ec .
The basic idea is to create a separate config sub-directory in .spyder2 for each environment from which Spyder is run. The name of the config directory is a hash of the interpreter path. Each installed Spyder will then run independent of the others, much in the same spirit as WinPython, while still using the user's HOME directory to store config. Highlights:
-
The first two commits are simply rearranging the bootstrap and startup processes so that all configuration happens before any import of spyder.py. If we do nothing else with this code I would at least recommend adopting these two changes. They make the bootstrap startup match the installed Spyder startup much better. It cleans up some ambiguity about when it's possible to ask for things like Qt binding versions and API selections. It also makes it much easier to move an entire configuration around during start_app.
-
The last three commits create the new environment-specific config sub-directory, remove the option to specify a separate Python interpreter for external consoles, and put the full path of the current interpreter into the main window title bar.
I have tested this on Python 2.7 on Windows 7 and Ubuntu 13.04, running two different environments on each. I used Python(x,y) and Anaconda on Windows. I used the standard system Python and Anaconda on Ubuntu. The two Spyder environments coexist peacefully together, and Spyder can be running in both simultaneously with no strange side effects. Most importantly, it requires essentially no user intervention to get this to happen smoothly.
Again, please consider this a prototype. If you want to provide code review, feel free. I would hope at this stage the focus of the review would be on the concept so we can arrive at a decision on which basic approach to adopt.
[1] https://groups.google.com/d/msg/spyderlib/-2OC4wOVB-o/THE8JRROAR4J [2]https://code.google.com/r/jedludlow-spyderlib-default/source/list?name=jedludlow-issue-1415
From ccordoba12 on 2013-07-13T15:19:38Z
Ok Jed, I'm glad you agree with me that we need to solve this issue as quickly as possible. Thanks a lot for providing a prototype solution for us to discuss on too, this way is easier to clarify our points of view.
These are my comments about your proposal:
-
As you said, your first two commits are really useful for they will let us to maintain separate configs for Python 2 and 3.
-
Unfortunately your proposal about having different configs per interpreter wouldn't work with our Mac app. The thing is the app can only be started using the embedded interpreter we ship it with (as far as I know).
But even if it'd be possible to do it with another one (running start_app from the terminal), the other interpreter would need to have PyQt/PySide installed, and Qt should be installed globally. Since the app comes with these libraries, it'd (kind of) defeat its purpose of not needing further installations.
- There could be another problem with it, well perhaps more of an annoyance. Let's suppose I spent a great deal of time configuring Spyder (shortcuts, color schemes, fonts, IPython, etc, etc). Then just by starting it with a different interpreter, all those settings would go away and I'd have to start from scratch again.
I'd be willing to accept this for Python 2 and 3 (because all people know they are incompatible) but not for virtualenvs of the same version. At the end I think people would copy their config from one dir to another (at least that's what I would do :-)
- So I think your proposal is sensible for scientific distributions but it's not scalable for Anaconda envs or virtualenvs, as my last point shows.
My feeling is that we need to go the PyDev route but without Projects. We don't need them right now, we just need to provide a good way to set and configure the interpreter used to evaluate code, while at the same time maintaining just one spyder.ini file (except for 2.7/3.3).
I'll start to develop my own approach (along the lines of comment #2) so we can also discuss its merits and faults.
From jed.lud...@gmail.com on 2013-07-13T19:00:27Z
My comments in reference to your numbered items:
-
I don't know the Mac app that well, but it seems redundant for Spyder to be shipping with its own interpreter and installed modules. I believe that most scientific Mac users will look to distributions like Anaconda to get a Python environment. We should optimize to work inside of those environments.
-
I would probably do exactly the same thing---copy the config to the new environment as a starting point. However, I actually think after the initial copy that I would want the two environments to each independently remember which files were open, breakpoints, maybe even custom window positions.
-
I actually agree :-). I think we should be targeting the broad majority of scientific users who install one, maybe two, Python environments, probably of different Python versions. Under your proposal, those users would still have to maintain two sets of configurations, one for 2.7 and another for 3.3. Those users who are sophisticated enough to be running multiple virtual environments are pretty comfortable with command line tools, vim, bash scripting, etc., and will probably have less interest in an IDE. Those that do want one can use PyDev. I'm okay with that.
From david.ve...@gmail.com on 2013-07-19T00:51:03Z
It is interesting to see how two strategies try to solve this problem in different ways :-) Just shooting out throwing some random, probably uniformed, ideas here.
Although I like the simplicity of Jed's idea, I would favour an approach whereby you can run Spyder with different interpreters. Point 1b of Carlos makes a lot of sense, and it is already True at my department. However, it seems that this approach can hold a lot of challenges and confusions (especially for less experienced users), mainly with respect to all the editor features (code completion, syntax checks) which might be tight to another environment. This is true for both devs and users alike. I don't know if that is possible or workable at all, but would it make sense to have both editor and console running off the same environment? Either with the same or different process/thread?
Maybe Jed's approach is useful regardless if Spyder wants to support running consoles from multiple environments or not. I obviously don't know how to tackle the Mac problems on Jed's plan, but to me it sounds as a good and simple enough idea to have in any case.
From jed.lud...@gmail.com on 2013-07-19T17:17:39Z
I pushed one more commit to my bookmark. The solution is now compatible with Python 3, too. I have tested this on Ubuntu 13.04 using Python 2.7 and 3.3. I have also tested it on Ubuntu 12.10 using Python 2.7 and 3.2. Sessions from different Python versions can co-exist peacefully.
CAVEAT: There is still an issue with the Project Explorer workspace pickling being incompatible between Python 2 and Python 3. That is a separate issue not yet addressed. So you can't have a workspace defined yet or you'll get crashes.
From jed.lud...@gmail.com on 2013-07-19T17:37:15Z
Sorry, that was testing with Ubuntu 12.04, not 12.10.
And the workspace problems are reported in issue #1414 .
From techtonik@gmail.com on 2013-07-20T03:10:15Z
How about documenting the approach to configs in SPEP? PySide took this approach from the Python and it is very useful.http://qt.gitorious.org/pyside/pseps/blobs/raw/34538e8bd11b6ba2b334d4a083a21f1e9209368c/psep-0100.txt It bears some overhead, but it pays back quickly as it helps future developers understand why current behavior is different from the most obvious one.
As for config problems, the ideal solution for me would be:
t1. Single spyder.ini with user preferences independent of Python
- user wants the same feeling of interface regardless of Python version
t2. Multiple directories containing Python interpreter configurations
- directories are located in spyder.ini dir
- directories are referenced from spyder.ini
- directory names are human friendly (no hashes)
- names can be created automatically (should carry meaning - why and what)
- spyder should complain about unreferenced interpreter configuration dirs
t3. There is a difference between
- Python interpreter to run Spyder
- Python interpreter to run things from Spyder
Depending on requirements for running Spyder and things running from Spyder, configurations for t4 can be further separated.
From techtonik@gmail.com on 2013-07-20T03:10:45Z
s/t4/t3/
From jed.lud...@gmail.com on 2013-09-09T07:21:36Z
Owner: ---
From ccordoba12 on 2013-10-03T08:19:19Z
Blocking: spyderlib:1365
From ccordoba12 on 2013-12-11T12:22:01Z
Labels: MS-v2.3.1
From ccordoba12 on 2013-12-11T12:24:00Z
Labels: -MS-v2.3
From ccordoba12 on 2014-07-14T07:10:15Z
Labels: -MS-v2.3.1 MS-v2.4
ccordoba12 removed the imported label on 10 Mar 2015
goanpeca was assigned by ccordoba12 on 13 Jul 2015
We need to tackle this one once the new projects arrive.
ccordoba12 removed the other label on 6 Janccordoba12 modified the milestone: v3.0, v3.0beta4 on 27 Janccordoba12 modified the milestone: v3.0beta4, v3.0rc1 on 26 Aprccordoba12 modified the milestone: v3.0beta4, v3.0rc1 on 10 Junccordoba12 modified the milestone: v3.1, v3.0rc1 on 4 Jul
From jed.lud...@gmail.com on 2013-05-31T00:43:05Z
It's now becoming more common to have multiple Python environments installed on a single system, and switching easily between them is becoming a requirement. The standard Python installers on Linux make this very easy to setup. Additionally, distributions like WinPython and Anaconda are making this easier on other platforms. With the possibility of multiple environments comes more ambiguity about Spyder user configuration.
Spyder stores user configuration in whatever directory HOME points to. With WinPython, Pierre has basically made each install a completely independent and portable environment, setting HOME to be a settings sub-directory within the WinPython install location. In that light, each Spyder instance (inside the various WinPython installs) has its own spyder.ini. However, WinPython is sort of unique in this regard. All the other environments leave HOME alone, which basically points them all back to the user's typical home location. This results in a shared spyder.ini and few resulting ambiguities.
Perhaps the most problematic setting is the path the Python interpreter to be used for external consoles. It makes it really messy to switch between environments. If you fire up Spyder from, say, the Anaconda Python 2.7 interpreter but you've already run Spyder before from another environment and pointed external consoles to use, say, c:\Python27\python.exe as the interpreter then Spyder comes up running in the Anaconda interpreter but it's external console is running somewhere else, importing modules from that environment. Maybe that's what you intended, but usually not. So you have to (1) realize that this is happening, (2) quickly jump to the Preferences dialog and fix it, and (3) probably restart Spyder just to be sure. For the new user this is a huge potential source of confusion. For the experienced user it's at the very least a hassle (I'm feeling it!). In the same spirit as issue #1402 , it makes me wonder if the ability to set the interpreter path is worth the potential confusion. I recognize that some Mac app users are taking advantage of this with EPD, but I still wonder if there isn't a cleaner way.
Switching between Python versions causes additional challenges as documented by issue #944 , usually requiring a config reset. Perhaps there is a way to come up with a unified sypder.ini that doesn't care about Python versions, but that sure seems like a tough order to fill given the number of changes to configparser between 2.7 and 3.3. Different spyder.ini files for each Python version, maybe?
Then you move on to lists of breakpoints, lists of open files, multiple Spyder instances all trying to update the same ini at the same time and it becomes less obvious how to handle all this across environments. There were enough difficult issues that I thought it at least worth a discussion about a design approach before experimenting with any changes.
Original issue: http://code.google.com/p/spyderlib/issues/detail?id=1415