Set up a python 3 virtual environment on CentOS 7.3

python virtual env

Python is a pretty useful language: powerful, easy to use and pretty much everything has a python port or interface available. The main drawback is that, as an interpreted language, it is not the fastest solution. But for the kind of “tinkering” that I do, ease of use really outranks speed.

Now since I prefer to use CentOS, using python version 3 takes a little setup and every time I do this, I need to look up something I forgot, so time for a Cheat-Sheet!

1. Install python version 3

Centos 7 still does not have a package for Python version 3. So it’s EPEL to the rescue: (you need to be root for this)

yum -y install epel-release
yum -y install python34

This should install python 3.4 alongside the existing version 2 of python:

python --version
Python 2.7.5
python3 --version
Python 3.4.5

2.Install Pip

Installing python packages goes much easier once you have pip, the “PyPA recommended tool for installing Python packages” installed. It’s quite easy to do if you have the EPEL repository enabled: (you need to be root for this)

yum -y install python-pip python-wheel

This should go quite quickly. If you don’t know python-wheel yet, its a ZIP-format archive format using the .whl extension and apparently, pip uses it.

To make sure that pip is up to date, use pip to upgrade itself: (you need to be root for this)

pip install -U pip
Collecting pip
 Downloading pip-9.0.1-py2.py3-none-any.whl (1.3MB)
 100% |....................................| 1.3MB 949kB/s 
Installing collected packages: pip
 Found existing installation: pip 8.1.2
 Uninstalling pip-8.1.2:
 Successfully uninstalled pip-8.1.2
Successfully installed pip-9.0.1

3. Install virtualenv

When playing with python on a dev machine, you’ll need to install lots of libraries which are used only by one or two projects. If you install everything in the global system locations, that will very quickly cause bloat and conflicts. The python standard way of dealing with this is to set up a virtual environment for each specific project.

Basically, it means setting up library paths to look at non globally installed libraries and the virtualenv package makes this quite easy to do. An added bonus is that using virtualenv and pip, you are able to set up your project environment as a non-root user, which makes things easier and safer.

First though we still need to install virtualenv and this is done easily using the pip command: (you need to be root for this)

pip install virtualenv
Collecting virtualenv
 Downloading virtualenv-15.1.0-py2.py3-none-any.whl (1.8MB)
 100% |........................................| 1.8MB 661kB/s 
Installing collected packages: virtualenv
Successfully installed virtualenv-15.1.0

4. Create a virtual environment

Now that all the moving parts are installed, we can go ahead and create a project environment. And as mentioned before, this is the point where we drop root privileges! Everything below is done as a regular user.

virtualenv -p python3 hbaserest
Running virtualenv with interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /home/jhon/hbaserest/bin/python3
Also creating executable in /home/jhon/hbaserest/bin/python
Installing setuptools, pip, wheel...done.

virtualenvstructThe “-p python3” option will make virtualenv create a python version 3 environment. If you leave this option out, you’ll end up with a python version 2 env. I am going to be working on an HBase Rest api project so I named my environment “hbaserest”.

As a result of running that command, the directory structure shown to the side was created.

The top three directories simply hold a local copy of the python libraries. Of course these are not real copies but mostly symbolic links to the global python environment under /usr/lib, /usr/lib64 and /usr/include. At this point, these directories are therefore quite small:

du -hs ./lin ./lib64 ./include
0      ./include
12M    ./lib
0      ./lib64

Once you start to install additional libraries for your project, those will be installed directly in these directories and at that point the sizes will increase.

Apart from those directories, there is the json file is just something pip uses for self-checking, as the name implies. The more interesting parts are in the bin directory. There you will the actual virtualenv “activate” script.

5. Using the virtual environment

Every time you want to start working in your virtual environment, you need to “activate” it, meaning you source the activate script to set the execution environment variables. It is recommended to source the script when inside the working directory.

cd hbaserest/
python --version
Python 2.7.5
source ./bin/activate
python --version
Python 3.4.5
deactivate
python --version
Python 2.7.5

I threw in a couple of “python –version” commands to demonstrate that the environment actually does change. Once you source the activate script, you will also see the command prompt changes to reflect the loaded virtual environment.

Once you’re done with the environment, you can use the “deactivate” function (which was added by the activate script) to undo the virtualenv environment changes.

6. Conclusion

So now the virtualenv is set up, simply remember to run the “source ./bin/activate” script every time you start working on your project and as long as you use the pip command, everything you add to the python environment should get added to the virtual environment, nicely isolated from the rest of the system.

Jhon Masschelein

Author: Jhon Masschelein

Tackler of advanced Cloud and Hadoop challenges in a world of open-source technologies. – Impossible is merely a matter of time and effort. –

2 thoughts on “Set up a python 3 virtual environment on CentOS 7.3”

  1. Hi Jhon,

    I am using CentOS7 and I have a default python version of Python 2.7.5. I want to install virtualenv and Python3.6. However, when I am trying your steps, I am getting the following error:

    $ sudo yum -y install epel-release
    Loaded plugins: amazon-id, product-id, rhui-lb, search-disabled-repos, subscription-manager
    This system is not registered with an entitlement server. You can use subscription-manager to register.
    Package epel-release-6-8.noarch already installed and latest version
    Nothing to do

    $ sudo yum -y install python34
    Loaded plugins: amazon-id, product-id, rhui-lb, search-disabled-repos, subscription-manager
    This system is not registered with an entitlement server. You can use subscription-manager to register.
    Resolving Dependencies
    –> Running transaction check
    —> Package python34.x86_64 0:3.4.8-1.el6 will be installed
    –> Processing Dependency: python34-libs(x86-64) = 3.4.8-1.el6 for package: python34-3.4.8-1.el6.x86_64
    –> Processing Dependency: libpython3.4m.so.1.0()(64bit) for package: python34-3.4.8-1.el6.x86_64
    –> Running transaction check
    —> Package python34-libs.x86_64 0:3.4.8-1.el6 will be installed
    –> Processing Dependency: liblzma.so.0()(64bit) for package: python34-libs-3.4.8-1.el6.x86_64
    –> Processing Dependency: libgdbm.so.2()(64bit) for package: python34-libs-3.4.8-1.el6.x86_64
    –> Processing Dependency: libffi.so.5()(64bit) for package: python34-libs-3.4.8-1.el6.x86_64
    –> Finished Dependency Resolution
    Error: Package: python34-libs-3.4.8-1.el6.x86_64 (epel)
    Requires: libgdbm.so.2()(64bit)
    Error: Package: python34-libs-3.4.8-1.el6.x86_64 (epel)
    Requires: liblzma.so.0()(64bit)
    Error: Package: python34-libs-3.4.8-1.el6.x86_64 (epel)
    Requires: libffi.so.5()(64bit)
    You could try using –skip-broken to work around the problem
    You could try running: rpm -Va –nofiles –nodigest

    $ sudo yum -y install python-pip
    Loaded plugins: amazon-id, product-id, rhui-lb, search-disabled-repos, subscription-manager
    This system is not registered with an entitlement server. You can use subscription-manager to register.
    Resolving Dependencies
    –> Running transaction check
    —> Package python-pip.noarch 0:7.1.0-1.el6 will be installed
    –> Processing Dependency: python(abi) = 2.6 for package: python-pip-7.1.0-1.el6.noarch
    –> Finished Dependency Resolution
    Error: Package: python-pip-7.1.0-1.el6.noarch (epel)
    Requires: python(abi) = 2.6
    Installed: python-2.7.5-69.el7_5.x86_64 (@rhui-REGION-rhel-server-releases)
    python(abi) = 2.7
    python(abi) = 2.7
    Available: python-2.7.5-16.el7.x86_64 (rhui-REGION-rhel-server-releases)
    python(abi) = 2.7
    python(abi) = 2.7
    Available: python-2.7.5-18.el7_1.1.x86_64 (rhui-REGION-rhel-server-releases)
    python(abi) = 2.7
    python(abi) = 2.7
    Available: python-2.7.5-34.el7.x86_64 (rhui-REGION-rhel-server-releases)
    python(abi) = 2.7
    python(abi) = 2.7
    Available: python-2.7.5-38.el7_2.x86_64 (rhui-REGION-rhel-server-releases)
    python(abi) = 2.7
    python(abi) = 2.7
    Available: python-2.7.5-39.el7_2.x86_64 (rhui-REGION-rhel-server-releases)
    python(abi) = 2.7
    python(abi) = 2.7
    Available: python-2.7.5-48.el7.x86_64 (rhui-REGION-rhel-server-releases)
    python(abi) = 2.7
    python(abi) = 2.7
    Available: python-2.7.5-58.el7.x86_64 (rhui-REGION-rhel-server-releases)
    python(abi) = 2.7
    python(abi) = 2.7
    Available: python-2.7.5-68.el7.x86_64 (rhui-REGION-rhel-server-releases)
    python(abi) = 2.7
    python(abi) = 2.7
    Available: python-2.7.5-76.el7.x86_64 (rhui-REGION-rhel-server-releases)
    python(abi) = 2.7
    python(abi) = 2.7
    Available: python34-3.4.8-1.el6.i686 (epel)
    python(abi) = 3.4
    You could try using –skip-broken to work around the problem
    You could try running: rpm -Va –nofiles –nodigest

    Can you please help me understand what am I missing here?

    Thanks in advance.

  2. This blog is a bit old now, so before debugging your output, I tried the install of Epel and python3.4 on a new CentOS7.4 system and that still work as described.

    Looking at your output, your Epel install seems to fail:

    $ sudo yum -y install epel-release
    Loaded plugins: amazon-id, product-id, rhui-lb, search-disabled-repos, subscription-manager
    This system is not registered with an entitlement server. You can use subscription-manager to register.
    Package epel-release-6-8.noarch already installed and latest version
    Nothing to do

    It tries to install epel-6-8 which is strange if yo are on Centos7. Epel releases have the same version number as the CentOS version you are trying to install on. (On my little test on CentOS7.4.1708, I got Epel release 7.11)

    I see that you have a plugin called amazon-id loaded… Are you trying this on an AWS VM?
    I have no experience with AWS (I use Azure) but I believe that the Linux versions on AWS are not “plain vanilla” so that may have something to do with the strange version request…

    But anyway, looking at the output, when you try to install the correct epel release, it tells you that Epel 6.8 is already installed, which is strange. Since the correct Epel release is never installed, the rest of the steps of course also fail.

    You could try to uninstall the epel-6-8 and then reinstall epel, hoping to get the CentOS7 version, but again, if you are on AWS, I have no idea how that will affect your VM.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.