Fixing Python 3.11 dependency breaks and Git authentication on Linux

Anyone running automation scripts on Linux servers knows that minor Python version upgrades can trigger a cascade of broken dependencies.

Recently, while running security and scanning utilities on an AlmaLinux server running Python 3.11 (such as collectivesqlscanner and holehe), I encountered a classic sequence of roadblocks: missing modules, breaking architectural refactors in lxml, version check exceptions, and interactive Git credential prompts blocking the terminal.

Here is the hands-on troubleshooting breakdown of how I diagnosed and fixed each issue.

1. The initial issue: ModuleNotFoundError: No module named 'requests_html'#

Running the main scanning script on Python 3.11:

python3.11 main.py -d "ver?id="

The script crashed immediately on import:

Traceback (most recent call last):
  File "/scripts/sql/collectivesqlscanner/main.py", line 1, in <module>
    from lib.sql import SQL
  File "/scripts/sql/collectivesqlscanner/lib/sql.py", line 2, in <module>
    from .search import Search
  File "/scripts/sql/collectivesqlscanner/lib/search.py", line 5, in <module>
    from requests_html import HTMLSession
ModuleNotFoundError: No module named 'requests_html'

Root cause and fix#

Python 3.11 did not have requests-html installed in its dedicated site-packages directory.

The straightforward fix was installing the package targeting the Python 3.11 environment:

pip3.11 install requests_html

2. The lxml architectural break: the lxml.html.clean case#

Once requests_html was installed, re-running the script triggered a breaking import exception inside lxml:

Traceback (most recent call last):
  File "/scripts/sql/collectivesqlscanner/main.py", line 1, in <module>
    from lib.sql import SQL
  File "/scripts/sql/collectivesqlscanner/lib/sql.py", line 2, in <module>
    from .search import Search
  File "/scripts/sql/collectivesqlscanner/lib/search.py", line 5, in <module>
    from requests_html import HTMLSession
  File "/usr/local/lib/python3.11/site-packages/requests_html.py", line 14, in <module>
    from lxml.html.clean import Cleaner
  File "/usr/local/lib64/python3.11/site-packages/lxml/html/clean.py", line 18, in <module>
    raise ImportError(
ImportError: lxml.html.clean module is now a separate project lxml_html_clean.
Install lxml[html_clean] or lxml_html_clean directly.

The lxml[html_clean] extra trap#

Following the error recommendation, I attempted to install the package extra:

pip3.11 install --upgrade "lxml[html_clean]"

Pip returned a warning:

Requirement already satisfied: lxml[html_clean] in /usr/local/lib64/python3.11/site-packages (6.0.1)
WARNING: lxml 6.0.1 does not provide the extra 'html_clean'

Re-running the script failed with the exact same ImportError.

Understanding the root cause#

In recent versions of lxml (such as 6.0.1), maintainers completely decoupled the clean module from the core library to minimize footprint and isolate HTML sanitization logic. The submodule now lives as an independent PyPI package: lxml_html_clean.

Because requests_html still makes the legacy import from lxml.html.clean import Cleaner, it requires this separate package to maintain backwards compatibility.

The definitive solution#

Installing the standalone package directly resolved the dependency tree:

pip3.11 install lxml_html_clean

With this package installed, Cleaner resolved cleanly in the expected namespace.

3. Version check failure in holehe (AttributeError: 'Response' object has no attribute 'json')#

Simultaneously, when running the OSINT email verification tool holehe, execution halted inside the trio async loop:

Traceback (most recent call last):
  File "/usr/local/bin/holehe", line 33, in <module>
    sys.exit(load_entry_point('holehe==1.61', 'console_scripts', 'holehe')())
  File "/usr/local/lib/python3.11/site-packages/holehe-1.61-py3.11.egg/holehe/core.py", line 233, in main
    trio.run(maincore)
  File "/usr/local/lib/python3.11/site-packages/trio-0.31.0-py3.11.egg/trio/_core/_run.py", line 2547, in run
    raise runner.main_task_outcome.error
  File "/usr/local/lib/python3.11/site-packages/holehe-1.61-py3.11.egg/holehe/core.py", line 197, in maincore
    check_update()
  File "/usr/local/lib/python3.11/site-packages/holehe-1.61-py3.11.egg/holehe/core.py", line 68, in check_update
    if check_version.json()["info"]["version"] != __version__:
       ^^^^^^^^^^^^^^^^^^
AttributeError: 'Response' object has no attribute 'json'

Technical analysis#

The check_update() function issues an HTTP GET request to PyPI to inspect whether a newer version of holehe is available.

If this request encounters firewall blocks, network timeouts, or redirects that yield a response object without the expected .json() method, the tool crashes during startup.

Workaround#

1. Verify Connectivity: Ensure the host resolves DNS and can reach PyPI endpoints via curl -I https://pypi.org/pypi/holehe/json.
2. Bypass Update Checks: Pass CLI flags to bypass automated startup version checks (such as --no-update or project-specific bypass flags).

4. Code synchronization and headless Git authentication#

Keeping tools aligned with upstream updates requires standard Git workflows:

• To clone a repository from scratch:

git clone https://github.com/example-user/example-repo.git

• To update an existing local clone:

git pull

The interactive prompt problem#

Running git pull on the server prompted for credentials:

[root@server:/scripts/sql/collectivesqlscanner]# git pull
Username for 'https://github.com':

Interactive credential prompts break headless scripts and automated cron jobs.

Reliable server authentication solutions#

Option 1: SSH Deploy Keys (Recommended Standard)#

Eliminates passwords and interactive prompts entirely.

1. Generate an ED25519 keypair on the host:

ssh-keygen -t ed25519 -C "user@domain.com"

1. Start the SSH agent and add the key:

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519

1. Display the public key to register in GitHub (Settings > SSH and GPG keys):

cat ~/.ssh/id_ed25519.pub

1. Switch the remote repository URL from HTTPS to SSH:

git remote set-url origin git@github.com:example-user/example-repo.git

Option 2: Personal Access Token (PAT)#

Because GitHub deprecated password authentication over HTTPS, generate a token under Developer settings > Personal access tokens. Use your GitHub username for the Username prompt and paste the PAT in place of the Password.

Option 3: Credential Helper Cache#

If you prefer keeping HTTPS transport:

git config --global credential.helper store

(Note: this stores credentials in plaintext inside ~/.git-credentials).

Best practices: isolating environments with venv#

Most dependency conflicts on enterprise Linux distributions (AlmaLinux, Rocky Linux, or Debian) stem from installing packages directly into the global system Python.

To protect scripts from cascading package breaks, isolate each tool inside a virtual environment:

# Navigate to the project directory
cd /scripts/sql/collectivesqlscanner/

# Create a virtual environment using Python 3.11
python3.11 -m venv venv

# Activate the virtual environment
source venv/bin/activate

# Install required packages in isolation
pip install requests_html lxml_html_clean

This ensures each tool maintains its own dependencies without polluting the host environment or breaking across OS updates.

See also

Development Migrating tools to Python 3.13 on Kali Linux: from TLS handshakes to c-api changes

Development Debugging the resolutionimpossible error in pip under Linux: Windows dependencies and practical workarounds

Cybersecurity Troubleshooting Fail2ban status 255 exception: migrating to systemd backend and Nginx modular includes

This post is licensed under CC BY-NC.