#pythonpath

Python et installation de pip3

Dans le cadre de l’administration et gestion de parc Windows, j’utilise des outils qui se configure ou utilisent python.

Sous Linux, il n’y a pratiquement rien à faire pour constituer son environnement de développement. Ce n’est pas le cas sous Windows. Je devais installer pip3 et ce n’est pas aussi simple.

(more…)

from distutils.sysconfig import get_python_lib

Adding Directories(packages) to PYTHONPATH has always been a pain.The runtime inclusion via python commands doesnt suit all the time.

Here is how you do it with a single entry in site-packages.My host was a Raspi.

cd /usr/local/lib/python2.X/site-packages

Say your package is foo and is hosted in /home/yourusername/yourdir/foo/myclass.py, create a file called foo.pth

In foo.pth add the path /home/yourusername/yourdir/foo/

How to: How do I find out my python path using python?

How do I find out my python path using python?

How do I find out which directories are listed in my system’s PYTHONPATH variable, from within a Python script (or the interactive shell)?

Answer: How do I find out my python path using python?

sys.path might include items that aren’t specifically in your PYTHONPATH environment variable. To query the variable directly, use:

import os try:…

Trying to get to the bottom of the best way to structure some code while working in virtualenv.

I needed to be clear on the use of modules:

Concretely, the import modu statement will look for the proper file, which is modu.py in the same directory as the caller if it exists. If it is not found, the Python interpreter will search for modu.py in the “path” recursively and raise an ImportError exception if it is not found. Once modu.py is found, the Python interpreter will execute the module in an isolated scope. Any top-level statement in modu.py will be executed, including other imports if any.

Then I needed to be clear that a bundle of modules may be referred to as a package:

Python provides a very straightforward packaging system, which is simply an extension of the module mechanism to a directory.

. However, some imports were failing although I had everything in packages as they should be. (Note "package" has two meanings in python, one being a directory of modules, the directory is named as the package, containing at least a blank __init__.py and submodules, but more usefullly may also contain subdirectories, each with their own __init__.py and submodules. (EDIT: found this good Mike Grouchy post about the use of __init__.py.) The other meaninghandy rule of thumb: import package-sense1, but pip install package-sense2

stackoverflow on editing PYTHONPATH in virtualenvs.

A post by mrcoles gives a pretty good idea of what you need to do re activate

Similar issue experienced by a stackoverflow user due to corruption of the name of

__init__.py

during SCP transfer

I ended up testing this via a tutorialspoint example.

While writing some code today for the XBRL project, I thought, "Writing the code to test this code is going to be a headache." Then, I realized I had been writing test code for years and deleting it when I was done with it. This motivated me to figure out one of python's testing libraries. 

I picked pytest because it looked simple. It is simple. After writing my first test, I ran into a problem with the customizing my PYTHONPATH variable on a project specific basis. I have the file structure:

projectdirectory\

normalmain.py

.modulea

.moduleb\

..test.py

..code_to_test.py (which imports modeula)

When I run pytest moduleb/test.py I get an error than modulea cannot be found! Since I normal program execution means running normalmain.py I want to keep the top level directory on my path.

I found a solution and also stumbled across a better way (for my usecase) to implement virtualenv than autowrapper in the process. Oh-my-zsh includes a plugin virtualenvwrapper which will automatically calls workon <virtualenv> when you enter a directory with a .venv file with the name of an existing virtualenv in it. This isn't anything to write home about, since autowrapper will do the same thing. But, when you to another directory that's not a subdirectory of the .venv directory, the Oh-my-zsh plugin will also call deactivate <virtualenv>. 

To implement the solution:

1. Create a .venv file with the project's virtualenv name in the project's top level directory. 

2. In the project's virtualenv directory, add the following to postactivate:

export OLD_PYTHONPATH=$PYTHONPATH

export $PYTHONPATH=<project top level directory>:$PYTHONPATH

3. In the project's vritualenv directory, add the following to the predeactivate:

export PYTHONPATH=$OLD_PYTHONPATH

Lately I've been working in virtualenvs which as been great for developing but not so great for installing. I've run into numerous issues where I prepend my PYTHONPATH with the directory I want to get imported first, but to no avail. You've run into this: you export PYTHONPATH exactly as you want, only to see that import sys; sys.path includes all kinds of other junk before it!

So, after watching a talk about reverse-engineering virtualenv. I realized I actually don't understand anything about what python does at startup. So after a lot of searching, I found out that to load packages, Python reads the site.py file. I read about site.py. And I found out.....

TURNS OUT PYTHON SECRETLY LOADS PACKAGES BEFORE LOOKING AT PYTHONPATH.

And where it looks is the *.pth files found in path/to/python/lib/site-packages/*.pth. The biggest culprit is usually easy_-nstall.pth. Mine had all kinds of absolute paths to the bigger Python install that I had to remove.

On my computer, easy-install.pth looks like this:

import sys; sys.__plen = len(sys.path) ./pytz-2013.8-py2.7.egg ./brewer2mpl-1.3.2-py2.7.egg ./pybedtools-0.6.2-py2.7-macosx-10.9-x86_64.egg /Users/olga/workspace-git/pandas /Users/olga/workspace-git/prettyplotlib /Users/olga/workspace-git/statsmodels /Users/olga/workspace-git/seaborn ./husl-2.1.0-py2.7.egg /Users/olga/workspace-git/gffutils ./simplejson-3.3.1-py2.7-macosx-10.9-x86_64.egg ./argcomplete-0.6.5-py2.7.egg ./argh-0.23.3-py2.7.egg ./moss-0.2.0-py2.7.egg /Users/olga/workspace-git/scikit-learn /Users/olga/workspace-git/matplotlib/lib ./Theano-0.6.0-py2.7.egg ./pysam-0.7.7-py2.7-macosx-10.9-x86_64.egg /Users/olga/workspace-git/YeoLab/gscripts import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p=getattr(sys,'__egginsert',0); sys.path[p:p]=new; sys.__egginsert = p+len(new)

So let's say I never wanted to use my development versions of something, then I'd remove that line from the file. Although eventually this got so much of a problem in my virtualenv at work that at I added the site-packages directory of that python distribution indirectly with ./:

import sys; sys.__plen = len(sys.path) ./ ./distribute-0.6.14-py2.7.egg /home/obotvinnik/workspace-git/gscripts /home/obotvinnik/workspace-git/rnaseqlib import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p=getattr(sys,'__egginsert',0); sys.path[p:p]=new; sys.__egginsert = p+len(new)

Because I always wanted to import from the virtualenv FIRST and never look at any other packages if I could help it.