Get up to 50% off on CKA, CKAD, CKS, KCNA, KCSA exams and courses!

Installing Python Modules in Disconnected Environments

Installing Python Modules in Disconnected Environments

Installing a python module or package (any applications) is very easy if you have internet connection and access to public repo or private repo.

Eg:

$ pip install pywinrm

But what if your production server do not have internet access and your repo server do not have this package to be installed ?

Here is the simple scenario; I am automating Window machines using Ansible . For Windows modules to work I need pywinrm module to be installed on the Ansible machine and I do not have internet access on Ansible.

Refer all Ansible Learning Guides here.

If the module or library do not have dependancies, then you simply download the package pypi.org/project/pywinrm and transfer this Ansible machine (or the machine which do not have internet access). But in our case, the pywinrm module have dependancies and we do not know what are the dependancies to be installed.

There is an easy method we can follow in this situation.

Identify the Python version on target machine

It is important to identify the target machine Python version and details before you download the libraries and packages.

Execute the following command to check the Python details on the target machine.

$ python -m pip debug --verbose
  • Identify Python version and ABI (examples: cp311, cp312).
  • Platform tag (examples: manylinux2014_x86_64 for most glibc-based Linux; musllinux for Alpine; win_amd64 for Windows).

E.g:

...
Compatible tags: 714
  cp311-cp311-manylinux_2_28_x86_64
  cp311-cp311-manylinux_2_27_x86_64
  cp311-cp311-manylinux_2_26_x86_64
...

Download the package from machine with internet access

From a machine which is connected to internet, download the packages and dependancies using pip download command. (We will download the package and dependencies to a directory to avoid mix-up)

Create a directory as repository to keep the packages and dependancies.

user@node1 $ mkdir pywinrm-local
pip download pywinrm \
  -d ./pywinrm-local  \
  --platform manylinux_2_17_x86_64 \
  --python-version 311 \
  --implementation cp \
  --abi cp311 \
  --only-binary=:all:

Note: If you have multiple files to download, then you can also use a requirements.txt and download them as follows.

# Download everything required by your requirements file
pip download -r ./requirements.txt \
  -d ./python-packages \
  --platform manylinux_2_17_x86_64 \
  --python-version 311 \
  --implementation cp \
  --abi cp311 \
  --only-binary=:all:

Now transfer the directory and content to target machine (disconnected machine) – the Ansible control node in our scenario – using any of the available methods. (scp, sftp, WinSCP etc)

user@node1 $ scp -rp pywinrm-local user@ansible-controlnode:/home/user/pywinrm-local

Install package on the disconnected machine

Verify the transferred packages and dependancy file; you will see all the .whl files inside.

user@ansible-controlnode $ ls -l pywinrm-local
total 6696
-rw-r--r--  1 user  user   149195  3 Jan 15:14 certifi-2021.10.8-py2.py3-none-any.whl
-rw-r--r--  1 user  user   178951  3 Jan 15:14 cffi-1.15.0-cp39-cp39-macosx_10_9_x86_64.whl
-rw-r--r--  1 user  user    39257  3 Jan 15:14 charset_normalizer-2.0.9-py3-none-any.whl
-rw-r--r--  1 user  user  2558077  3 Jan 15:14 cryptography-36.0.1-cp36-abi3-macosx_10_10_x86_64.whl
-rw-r--r--  1 user  user    61160  3 Jan 15:14 idna-3.3-py3-none-any.whl
-rw-r--r--  1 user  user    29980  3 Jan 15:14 ntlm_auth-1.5.0-py2.py3-none-any.whl
-rw-r--r--  1 user  user   118697  3 Jan 15:14 pycparser-2.21-py2.py3-none-any.whl
-rw-r--r--  1 user  user    44274  3 Jan 15:14 pywinrm-0.4.2-py2.py3-none-any.whl
-rw-r--r--  1 user  user    62251  3 Jan 15:14 requests-2.26.0-py2.py3-none-any.whl
-rw-r--r--  1 user  user     5717  3 Jan 15:14 requests_ntlm-1.1.0-py2.py3-none-any.whl
-rw-r--r--  1 user  user    11053  3 Jan 15:14 six-1.16.0-py2.py3-none-any.whl
-rw-r--r--  1 user  user   138764  3 Jan 15:14 urllib3-1.26.7-py2.py3-none-any.whl
-rw-r--r--  1 user  user     9170  3 Jan 15:14 xmltodict-0.12.0-py2.py3-none-any.whl

Create a requirements.txt file with all filenames in the repo directory (eg: pywinrm-local )

user@ansible-controlnode $ ls -l pywinrm-local |awk '{print $9}' > pywinrm-local/requirements.txt

Now Install the package using pip command by specifying the source repo directory.

user@ansible-controlnode $ pip install \
  -r pywinrm-local/requirements.txt \
  --find-links=pywinrm-local \
  --no-index

Verify installed Python module

user@ansible-controlnode $ pip list |grep pywinrm
pywinrm            0.4.2

You can use this method for any complex python deployments but just need to make sure all the dependancies are downloaded to the same repo directory.

Gineesh Madapparambath

Gineesh Madapparambath

Gineesh Madapparambath is the founder of techbeatly. He is the co-author of The Kubernetes Bible, Second Edition and the author of Ansible for Real Life Automation. He has worked as a Systems Engineer, Automation Specialist, and content author. His primary focus is on Ansible Automation, Containerisation (OpenShift & Kubernetes), and Infrastructure as Code (Terraform). (Read more: iamgini.com)


Note

Disclaimer: The views expressed and the content shared in all published articles on this website are solely those of the respective authors, and they do not necessarily reflect the views of the author’s employer or the techbeatly platform. We strive to ensure the accuracy and validity of the content published on our website. However, we cannot guarantee the absolute correctness or completeness of the information provided. It is the responsibility of the readers and users of this website to verify the accuracy and appropriateness of any information or opinions expressed within the articles. If you come across any content that you believe to be incorrect or invalid, please contact us immediately so that we can address the issue promptly.

Share :

Related Posts

Managing Ansible Inventory

Managing Ansible Inventory

Inventory File An inventory file is a simple and plain text file where we store those host information ansible has to deal with. It can be simple in …

Implementing Ansible Roles

Implementing Ansible Roles

In a production environment, we will have hundreds and thousands of different types machines to handle. Hence the type of operations and …

Deploying Roles With Ansible Galaxy

Deploying Roles With Ansible Galaxy

So you have learned about Ansible Roles and the importance of using roles. Ansible Galaxy is a public library where you can find thousands of roles …