Skip to content

Remote: Automatically manage known hosts keys#451

Merged
jan-janssen merged 6 commits intomainfrom
known_hosts
Apr 11, 2026
Merged

Remote: Automatically manage known hosts keys#451
jan-janssen merged 6 commits intomainfrom
known_hosts

Conversation

@jan-janssen
Copy link
Copy Markdown
Member

@jan-janssen jan-janssen commented Aug 13, 2025

Summary by CodeRabbit

  • Bug Fixes

    • Refined SSH host key handling for remote queue adapters to support optional known_hosts configuration with intelligent automatic fallback for unrecognized hosts.
  • Tests

    • Added and updated test coverage for SSH continuous connection behavior, multiple authentication methods, and job operations with revised test configuration fixtures.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Aug 13, 2025

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 34c9658a-676d-4942-bdf8-d719571f05ab

📥 Commits

Reviewing files that changed from the base of the PR and between 8868a61 and ccf1586.

📒 Files selected for processing (5)
  • src/pysqa/base/remote.py
  • tests/static/remote_rebex/queue.yaml
  • tests/static/remote_rebex_hosts/queue.yaml
  • tests/unit/base/test_remote.py
  • tests/unit/base/test_remote_auth.py

📝 Walkthrough

Walkthrough

SSH host key handling in RemoteQueueAdapter is now conditional: the known_hosts configuration entry is optional, and when absent, the adapter falls back to using Paramiko's AutoAddPolicy instead of loading host keys from a file.

Changes

Cohort / File(s) Summary
SSH Connection Configuration
src/pysqa/base/remote.py
Made known_hosts optional during adapter initialization; host key handling now branches: loads host keys when known_hosts exists, applies AutoAddPolicy fallback when absent.
Test Fixtures
tests/static/remote_rebex/queue.yaml, tests/static/remote_rebex_hosts/queue.yaml
Removed known_hosts reference from existing fixture; added new fixture with known_hosts explicitly configured.
Authentication & Integration Tests
tests/unit/base/test_remote.py, tests/unit/base/test_remote_auth.py
Updated all test cases to reference new fixture directory (remote_rebex_hosts); added two new tests for continuous-connection behavior with conditional known_hosts handling.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Possibly related PRs

Poem

🐰 A rabbit hops through SSH gates so fine,
Where known-hosts were required in days of yore.
Now optional they sit, a gentle line—
When absent, AutoPolicy opens the door! 🔑

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch known_hosts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov
Copy link
Copy Markdown

codecov bot commented Aug 13, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.50%. Comparing base (3ab449a) to head (ccf1586).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #451      +/-   ##
==========================================
+ Coverage   92.48%   92.50%   +0.02%     
==========================================
  Files          20       20              
  Lines        1064     1068       +4     
==========================================
+ Hits          984      988       +4     
  Misses         80       80              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🔭 Outside diff range comments (1)
pysqa/base/remote.py (1)

448-451: Inconsistent and insecure host-key policy for proxy connection

For the proxy (bastion) connection, AutoAddPolicy is hardcoded, bypassing the known_hosts behavior and opt-in policy you established earlier. This is a security hole and inconsistent with the rest of the connection setup.

Align the proxy client policy with the main client:

-            client_new = paramiko.SSHClient()
-            client_new.set_missing_host_key_policy(paramiko.AutoAddPolicy())
+            client_new = paramiko.SSHClient()
+            client_new.load_system_host_keys()
+            if self._ssh_known_hosts is not None:
+                try:
+                    client_new.load_host_keys(self._ssh_known_hosts)
+                except FileNotFoundError:
+                    warnings.warn(
+                        f"known_hosts file not found at '{self._ssh_known_hosts}'. Using system host keys only.",
+                        stacklevel=2,
+                    )
+            if getattr(self, "_ssh_auto_add_unknown_hosts", False):
+                client_new.set_missing_host_key_policy(paramiko.AutoAddPolicy())
+            else:
+                client_new.set_missing_host_key_policy(paramiko.RejectPolicy())
🧹 Nitpick comments (2)
pysqa/base/remote.py (2)

28-35: Update attributes docstring to reflect Optional known_hosts and new policy flag

doc drift: _ssh_known_hosts can now be None; document it. Also document the new ssh_auto_add_unknown_hosts flag if you adopt it.

Apply this doc update:

-        _ssh_known_hosts (str): The path to the known hosts file.
+        _ssh_known_hosts (Optional[str]): Path to the known hosts file. None uses system host keys.
+        _ssh_auto_add_unknown_hosts (bool): If True, auto-accept and add unknown host keys (unsafe by default).

351-469: Optional: Persist auto-added host keys to disk when a known_hosts path is configured

Right now, AutoAddPolicy only amends the in-memory host keys for the client session. If you intend to "automatically manage known hosts keys," consider persisting newly seen keys to the configured known_hosts file so future sessions verify them.

You can add a small helper and call it after each successful ssh.connect(...):

def _persist_server_key(self, ssh: paramiko.SSHClient) -> None:
    if not getattr(self, "_ssh_auto_add_unknown_hosts", False):
        return
    if not self._ssh_known_hosts:
        return
    transport = get_transport(ssh)
    server_key = transport.get_remote_server_key()
    hostkeys = paramiko.HostKeys()
    if os.path.exists(self._ssh_known_hosts):
        hostkeys.load(self._ssh_known_hosts)
    hostkeys.add(self._ssh_host, server_key.get_name(), server_key)
    os.makedirs(os.path.dirname(self._ssh_known_hosts), exist_ok=True)
    hostkeys.save(self._ssh_known_hosts)

Then, right after each ssh.connect(...) call in _open_ssh_connection, invoke:

self._persist_server_key(ssh)

And similarly after client_new.connect(...) for the proxy.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 8051c3d and 0cf2c65.

📒 Files selected for processing (1)
  • pysqa/base/remote.py (2 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
  • GitHub Check: unittest_matrix (windows-latest, 3.13)
  • GitHub Check: unittest_slurm

Comment on lines +96 to +101
if "known_hosts" in config:
self._ssh_known_hosts = os.path.abspath(
os.path.expanduser(config["known_hosts"])
)
else:
self._ssh_known_hosts = None
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Harden known_hosts config handling; avoid empty-string bug and add explicit opt-in for auto-accepting unknown hosts

  • Using "key in config" treats an empty string as a valid path and resolves it to the current working directory, causing paramiko to try to read a non-existent file.
  • Defaulting to AutoAddPolicy downstream without explicit opt-in is a security regression; prefer system host keys by default and make auto-accept a config flag.

Proposed fix:

  • Treat empty or missing known_hosts as None.
  • Validate file existence and warn if missing.
  • Introduce ssh_auto_add_unknown_hosts (default False).

Apply this diff:

-        if "known_hosts" in config:
-            self._ssh_known_hosts = os.path.abspath(
-                os.path.expanduser(config["known_hosts"])
-            )
-        else:
-            self._ssh_known_hosts = None
+        # Known hosts: allow None, but prefer system host keys when unspecified.
+        kh = config.get("known_hosts")
+        if kh:
+            path = os.path.abspath(os.path.expanduser(kh))
+            if os.path.exists(path):
+                self._ssh_known_hosts = path
+            else:
+                warnings.warn(
+                    f"known_hosts file not found at '{path}'. Falling back to system host keys.",
+                    stacklevel=2,
+                )
+                self._ssh_known_hosts = None
+        else:
+            self._ssh_known_hosts = None
+        # Opt-in flag to auto-accept unknown host keys (unsafe by default).
+        self._ssh_auto_add_unknown_hosts = bool(
+            config.get("ssh_auto_add_unknown_hosts", False)
+        )
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if "known_hosts" in config:
self._ssh_known_hosts = os.path.abspath(
os.path.expanduser(config["known_hosts"])
)
else:
self._ssh_known_hosts = None
# Known hosts: allow None, but prefer system host keys when unspecified.
kh = config.get("known_hosts")
if kh:
path = os.path.abspath(os.path.expanduser(kh))
if os.path.exists(path):
self._ssh_known_hosts = path
else:
warnings.warn(
f"known_hosts file not found at '{path}'. Falling back to system host keys.",
stacklevel=2,
)
self._ssh_known_hosts = None
else:
self._ssh_known_hosts = None
# Opt-in flag to auto-accept unknown host keys (unsafe by default).
self._ssh_auto_add_unknown_hosts = bool(
config.get("ssh_auto_add_unknown_hosts", False)
)
🤖 Prompt for AI Agents
In pysqa/base/remote.py around lines 96 to 101, the current known_hosts handling
treats an empty string as a valid path and implicitly enables AutoAddPolicy
downstream; change it to treat missing or empty known_hosts as None, add a new
boolean config key ssh_auto_add_unknown_hosts defaulting to False, and only
enable auto-accept when that flag is true. Specifically: read
config.get("known_hosts") and if value is falsy (None or empty string) set
self._ssh_known_hosts = None; if a non-empty path is provided expand and abspath
it and then check os.path.exists(path) — if it does not exist set
self._ssh_known_hosts = None and emit a warning via the module logger; add
self._ssh_auto_add_unknown_hosts = bool(config.get("ssh_auto_add_unknown_hosts",
False)) so callers can decide whether to set AutoAddPolicy; ensure default
behavior prefers loading system host keys when known_hosts is None and only use
AutoAddPolicy when the new flag is true.

Comment on lines +352 to +355
if self._ssh_known_hosts is not None:
ssh.load_host_keys(self._ssh_known_hosts)
else:
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Avoid silently trusting unknown hosts; load system host keys and gate AutoAddPolicy behind explicit opt-in

When _ssh_known_hosts is None, switching to AutoAddPolicy unconditionally trusts unknown hosts, enabling MITM risk. Prefer:

  • Always load system host keys.
  • Optionally load a user-provided known_hosts file.
  • Use RejectPolicy by default; only AutoAdd when explicitly configured.

Apply this diff:

-        if self._ssh_known_hosts is not None:
-            ssh.load_host_keys(self._ssh_known_hosts)
-        else:
-            ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
+        # Load system-level keys first; optionally supplement with a custom file.
+        ssh.load_system_host_keys()
+        if self._ssh_known_hosts is not None:
+            try:
+                ssh.load_host_keys(self._ssh_known_hosts)
+            except FileNotFoundError:
+                warnings.warn(
+                    f"known_hosts file not found at '{self._ssh_known_hosts}'. Using system host keys only.",
+                    stacklevel=2,
+                )
+        # Decide how to handle unknown hosts
+        if getattr(self, "_ssh_auto_add_unknown_hosts", False):
+            ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
+        else:
+            ssh.set_missing_host_key_policy(paramiko.RejectPolicy())

Committable suggestion skipped: line range outside the PR's diff.

🤖 Prompt for AI Agents
In pysqa/base/remote.py around lines 352-355, the code currently sets
AutoAddPolicy when self._ssh_known_hosts is None which silently trusts unknown
hosts; change it to always call ssh.load_system_host_keys(), then if
self._ssh_known_hosts is set load that file too, and set the missing host key
policy to paramiko.RejectPolicy() by default; introduce or use an explicit
boolean config flag (e.g. self._allow_auto_add_hosts defaulting to False) so
AutoAddPolicy() is only applied when that flag is True (otherwise use
RejectPolicy()).

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (3)
tests/test_remote_auth.py (1)

22-22: Switching to host-based config looks correct; consider adding coverage for the "no known_hosts" path.

The updated QueueAdapter(directory=.../config/remote_rebex_hosts) is consistent with making known_hosts optional. To fully exercise the new behavior, add a test that sets _ssh_known_hosts to "" and asserts AutoAddPolicy is applied (without making a real network call).

Add this test method to TestRemoteQueueAdapterAuth:

def test_known_hosts_optional_auto_add_policy(self):
    # Ensures that when _ssh_known_hosts is empty, AutoAddPolicy is set
    from unittest.mock import patch

    class FakeSSHClient:
        def __init__(self, *args, **kwargs):
            self.auto_add_policy_set = False
            self.host_keys_loaded = False

        def load_host_keys(self, *args, **kwargs):
            self.host_keys_loaded = True

        def set_missing_host_key_policy(self, policy):
            # We don't depend on instance type here; just record that it was set
            self.auto_add_policy_set = True

        def connect(self, *args, **kwargs):
            # No-op to avoid real network calls
            pass

    path = os.path.dirname(os.path.abspath(__file__))
    remote = QueueAdapter(directory=os.path.join(path, "config/remote_rebex_hosts"))

    # Force the "no known_hosts" branch and a password-based auth path
    remote._adapter._ssh_known_hosts = ""
    remote._adapter._ssh_key = None
    remote._adapter._ssh_ask_for_password = False
    remote._adapter._ssh_password = remote._adapter._ssh_password or "dummy"

    with patch("pysqa.base.remote.paramiko.SSHClient", FakeSSHClient):
        ssh = remote._adapter._open_ssh_connection()
        # Validate behavior
        self.assertTrue(ssh.auto_add_policy_set)
        self.assertFalse(ssh.host_keys_loaded)

Also applies to: 29-29, 36-36

tests/test_remote.py (2)

162-170: Nit: fix typo in test name and ensure the persistent connection is closed.

  • Typo: “continous” → “continuous” in the test name.
  • Since you explicitly open a persistent connection, close it in a finally block to avoid leaking connections in CI.

Apply this diff:

-def test_remote_command_continous_connection_hosts(self):
+def test_remote_command_continuous_connection_hosts(self):
     path = os.path.dirname(os.path.abspath(__file__))
     remote = QueueAdapter(directory=os.path.join(path, "config/remote_rebex_hosts"))
     remote._adapter._ssh_remote_path = path
     remote._adapter._ssh_continous_connection = True
-    remote._adapter._open_ssh_connection()
-    output = remote._adapter._execute_remote_command(command="pwd")
-    self.assertEqual(output, "/\n")
+    ssh = remote._adapter._open_ssh_connection()
+    try:
+        output = remote._adapter._execute_remote_command(command="pwd")
+        self.assertEqual(output, "/\n")
+    finally:
+        if ssh is not None:
+            ssh.close()
+            remote._adapter._ssh_connection = None

Note: The attribute name _ssh_continous_connection is misspelled in production code; keep it as-is here to avoid breaking behavior. If you plan to fix the spelling in code later, deprecate with a compatibility shim.


173-173: LGTM on switching remaining tests to host-based config; consider DRYing repeated path construction.

All these tests now initialize via config/remote_rebex_hosts, which aligns with the new known_hosts handling. To reduce duplication and improve maintainability, compute the hosts config path once in setUpClass or setUp of TestRemoteQueueAdapterRebex and reuse it.

Example refactor (outside this hunk):

@classmethod
def setUpClass(cls):
    cls.path = os.path.dirname(os.path.abspath(__file__))
    cls.hosts_dir = os.path.join(cls.path, "config/remote_rebex_hosts")

def _make_remote(self):
    return QueueAdapter(directory=self.hosts_dir)

# Usage in tests:
remote = self._make_remote()

Alternatively, parameterize over both config/remote_rebex and config/remote_rebex_hosts using subTest to collapse duplicated tests.

Also applies to: 180-180, 186-186, 193-193, 200-200

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 41a42a0 and 8868a61.

📒 Files selected for processing (3)
  • tests/config/remote_rebex_hosts/queue.yaml (1 hunks)
  • tests/test_remote.py (1 hunks)
  • tests/test_remote_auth.py (1 hunks)
✅ Files skipped from review due to trivial changes (1)
  • tests/config/remote_rebex_hosts/queue.yaml
🧰 Additional context used
🧬 Code Graph Analysis (2)
tests/test_remote_auth.py (2)
pysqa/queueadapter.py (1)
  • QueueAdapter (13-384)
pysqa/base/remote.py (1)
  • _open_ssh_connection (344-468)
tests/test_remote.py (2)
pysqa/queueadapter.py (2)
  • QueueAdapter (13-384)
  • submit_job (188-231)
pysqa/base/remote.py (5)
  • _open_ssh_connection (344-468)
  • _execute_remote_command (559-582)
  • submit_job (147-183)
  • transfer_file (266-291)
  • get_transport (680-684)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: unittest_matrix (windows-latest, 3.13)
  • GitHub Check: unittest_slurm
  • GitHub Check: unittest_matrix (macos-latest, 3.13)

Comment on lines +154 to +161
def test_remote_command_individual_connections_hosts(self):
path = os.path.dirname(os.path.abspath(__file__))
remote = QueueAdapter(directory=os.path.join(path, "config/remote_rebex_hosts"))
remote._adapter._ssh_remote_path = path
remote._adapter._open_ssh_connection()
output = remote._adapter._execute_remote_command(command="pwd")
self.assertEqual(output, "/\n")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Avoid opening a redundant SSH connection in the individual-connections host test.

When _ssh_continous_connection is False, _execute_remote_command opens and closes its own connection. Calling _open_ssh_connection beforehand (Line 158) creates an unused connection and may leak resources.

Apply this diff:

 def test_remote_command_individual_connections_hosts(self):
     path = os.path.dirname(os.path.abspath(__file__))
     remote = QueueAdapter(directory=os.path.join(path, "config/remote_rebex_hosts"))
     remote._adapter._ssh_remote_path = path
-    remote._adapter._open_ssh_connection()
     output = remote._adapter._execute_remote_command(command="pwd")
     self.assertEqual(output, "/\n")

Optionally, make the same change in test_remote_command_individual_connections above (Lines 141–143) for consistency.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
def test_remote_command_individual_connections_hosts(self):
path = os.path.dirname(os.path.abspath(__file__))
remote = QueueAdapter(directory=os.path.join(path, "config/remote_rebex_hosts"))
remote._adapter._ssh_remote_path = path
remote._adapter._open_ssh_connection()
output = remote._adapter._execute_remote_command(command="pwd")
self.assertEqual(output, "/\n")
def test_remote_command_individual_connections_hosts(self):
path = os.path.dirname(os.path.abspath(__file__))
remote = QueueAdapter(directory=os.path.join(path, "config/remote_rebex_hosts"))
remote._adapter._ssh_remote_path = path
output = remote._adapter._execute_remote_command(command="pwd")
self.assertEqual(output, "/\n")
🤖 Prompt for AI Agents
In tests/test_remote.py around lines 154 to 161, the test calls
remote._adapter._open_ssh_connection() before executing the command which is
redundant when _ssh_continous_connection is False and causes a leaked unused
connection; remove the explicit call to _open_ssh_connection() (line 158) so
_execute_remote_command manages its own connect/disconnect, and also apply the
same removal to the similar test at lines ~141–143 for consistency.

@jan-janssen jan-janssen marked this pull request as draft February 16, 2026 09:48
@jan-janssen jan-janssen marked this pull request as ready for review April 11, 2026 06:45
Copilot AI review requested due to automatic review settings April 11, 2026 06:45
@jan-janssen jan-janssen merged commit cd4c8b0 into main Apr 11, 2026
27 of 28 checks passed
@jan-janssen jan-janssen deleted the known_hosts branch April 11, 2026 06:46
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR makes SSH known-hosts handling optional for RemoteQueueAdapter: if no known_hosts is provided in the queue config, remote connections will accept unknown host keys automatically to simplify first-time connections, while preserving the existing behavior when known_hosts is configured.

Changes:

  • Make known_hosts optional in RemoteQueueAdapter and fall back to paramiko.AutoAddPolicy() when absent.
  • Update Rebex remote test fixtures to cover both “no known_hosts” and “known_hosts present” configurations.
  • Expand unit/integration tests to exercise host-based remote configurations.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
src/pysqa/base/remote.py Makes known_hosts optional and changes host-key policy selection during SSH connection setup.
tests/unit/base/test_remote.py Adds host-key-related Rebex tests and switches multiple Rebex tests to a config that includes known_hosts.
tests/unit/base/test_remote_auth.py Switches auth tests to use the Rebex config that includes known_hosts.
tests/static/remote_rebex/queue.yaml Removes known_hosts to exercise the new optional behavior.
tests/static/remote_rebex_hosts/queue.yaml Adds a new fixture that includes known_hosts (currently points to ~/.ssh/known_hosts).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +96 to +98
if "known_hosts" in config:
self._ssh_known_hosts = os.path.abspath(
os.path.expanduser(config["known_hosts"])
Copy link

Copilot AI Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

known_hosts is treated as present solely based on the key existing in config. If a user sets known_hosts: "" (or it gets parsed as an empty string), abspath(expanduser("")) becomes the current working directory and load_host_keys() will later try to read a directory path. Consider using known_hosts = config.get("known_hosts") and only setting _ssh_known_hosts when it is a non-empty path (otherwise treat it as unset).

Suggested change
if "known_hosts" in config:
self._ssh_known_hosts = os.path.abspath(
os.path.expanduser(config["known_hosts"])
known_hosts = config.get("known_hosts")
if known_hosts:
self._ssh_known_hosts = os.path.abspath(
os.path.expanduser(known_hosts)

Copilot uses AI. Check for mistakes.
Comment on lines +352 to +355
if len(self._ssh_known_hosts) > 0:
ssh.load_host_keys(self._ssh_known_hosts)
else:
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
Copy link

Copilot AI Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When known_hosts is unset this enables AutoAddPolicy(), which silently trusts the first presented host key (MITM risk). Consider at least emitting a warning (and/or using WarningPolicy) so users know host key verification is disabled. Also, when known_hosts is set, load_host_keys() will raise if the file doesn’t exist; it would be more robust to os.path.exists()-check and fall back to the missing-host-key policy with a warning rather than crashing.

Suggested change
if len(self._ssh_known_hosts) > 0:
ssh.load_host_keys(self._ssh_known_hosts)
else:
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
if len(self._ssh_known_hosts) > 0 and os.path.exists(self._ssh_known_hosts):
ssh.load_host_keys(self._ssh_known_hosts)
else:
if len(self._ssh_known_hosts) > 0:
warnings.warn(
"Configured SSH known_hosts file does not exist: "
+ self._ssh_known_hosts
+ ". Host key verification will use WarningPolicy instead.",
UserWarning,
)
else:
warnings.warn(
"SSH known_hosts is not configured. Host key verification is "
"disabled for unknown hosts and WarningPolicy will be used.",
UserWarning,
)
ssh.set_missing_host_key_policy(paramiko.WarningPolicy())

Copilot uses AI. Check for mistakes.
Comment on lines +155 to +159
path = os.path.dirname(os.path.abspath(__file__))
remote = QueueAdapter(directory=os.path.join(path, "../../static/remote_rebex_hosts"))
remote._adapter._ssh_remote_path = path
remote._adapter._open_ssh_connection()
output = remote._adapter._execute_remote_command(command="pwd")
Copy link

Copilot AI Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These new "_hosts" integration tests depend on the runner’s ~/.ssh/known_hosts containing the correct key for test.rebex.net. That makes CI/local runs brittle and environment-dependent. Prefer using a repo-provided known_hosts fixture (or a temp file created during the test), or mock Paramiko’s host-key handling so the test doesn’t rely on a developer/CI home directory.

Copilot uses AI. Check for mistakes.
Comment on lines 171 to 175
def test_submit_job(self):
path = os.path.dirname(os.path.abspath(__file__))
remote = QueueAdapter(directory=os.path.join(path, "../../static/remote_rebex"))
remote = QueueAdapter(directory=os.path.join(path, "../../static/remote_rebex_hosts"))
remote._adapter._ssh_remote_path = path
output = remote._adapter.submit_job(working_directory=os.path.join(path, "../../static/empty"), command="echo 1")
Copy link

Copilot AI Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This (and the other Rebex tests below) now use remote_rebex_hosts, which requires ~/.ssh/known_hosts to exist and include the Rebex host key. That will likely fail on clean CI images and reduces coverage of the new behavior where known_hosts is omitted. Consider switching these back to remote_rebex (no known_hosts) and keeping host-key verification behavior covered via a deterministic fixture or a mocked Paramiko client.

Copilot uses AI. Check for mistakes.
Comment on lines 178 to 182
def test_transferfile_individual_connections(self):
path = os.path.dirname(os.path.abspath(__file__))
remote = QueueAdapter(directory=os.path.join(path, "../../static/remote_rebex"))
remote = QueueAdapter(directory=os.path.join(path, "../../static/remote_rebex_hosts"))
remote._adapter._ssh_remote_path = path
self.assertIsNone(remote._adapter.transfer_file(file="readme.txt", transfer_back=True))
Copy link

Copilot AI Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using remote_rebex_hosts here makes the test dependent on an external ~/.ssh/known_hosts file (which may not exist in CI). Either use the config without known_hosts for these integration tests, or supply a known_hosts fixture under tests/static and reference it from the YAML.

Copilot uses AI. Check for mistakes.
Comment on lines 191 to 194
def test_get_transport(self):
path = os.path.dirname(os.path.abspath(__file__))
remote = QueueAdapter(directory=os.path.join(path, "../../static/remote_rebex"))
remote = QueueAdapter(directory=os.path.join(path, "../../static/remote_rebex_hosts"))
self.assertIsNotNone(get_transport(remote._adapter._open_ssh_connection()))
Copy link

Copilot AI Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test will be flaky in environments where ~/.ssh/known_hosts is absent or doesn’t contain test.rebex.net. For stability, avoid relying on the user/CI home directory: use a repo-local known_hosts fixture or mock Paramiko’s host key checks/policies.

Copilot uses AI. Check for mistakes.
Comment on lines 199 to 203
path = os.path.dirname(os.path.abspath(__file__))
remote = QueueAdapter(directory=os.path.join(path, "../../static/remote_rebex"))
remote = QueueAdapter(directory=os.path.join(path, "../../static/remote_rebex_hosts"))
remote._adapter._ssh_remote_path = path
remote._adapter._ssh_local_path = path
remote._adapter._ssh_delete_file_on_remote = True
Copy link

Copilot AI Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue as above: remote_rebex_hosts makes this unit test suite depend on ~/.ssh/known_hosts contents. Consider using remote_rebex (no known_hosts) for functional coverage and cover the known_hosts path via a deterministic fixture/mock so tests remain hermetic.

Copilot uses AI. Check for mistakes.
Comment on lines 20 to 25
def test_password_auth(self):
path = os.path.dirname(os.path.abspath(__file__))
remote = QueueAdapter(directory=os.path.join(path, "../../static/remote_rebex"))
remote = QueueAdapter(directory=os.path.join(path, "../../static/remote_rebex_hosts"))
remote._adapter._ssh_ask_for_password = False
remote._adapter._ssh_key = None
self.assertIsNotNone(remote._adapter._open_ssh_connection())
Copy link

Copilot AI Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All auth tests now use remote_rebex_hosts, which makes the suite depend on ~/.ssh/known_hosts existing and containing the Rebex host key. That’s not guaranteed in CI and can cause unrelated failures. Prefer using the config without known_hosts, or point known_hosts at a repo-local fixture file for deterministic behavior.

Copilot uses AI. Check for mistakes.
queue_primary: remote
ssh_host: test.rebex.net
ssh_username: demo
known_hosts: ~/.ssh/known_hosts
Copy link

Copilot AI Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pointing known_hosts at ~/.ssh/known_hosts makes tests/configs environment-dependent (often missing on CI, and contents vary per machine). For stable tests, prefer a repository-local known_hosts fixture file (checked into tests/static/...) or omit known_hosts in this test config and mock host-key verification when needed.

Suggested change
known_hosts: ~/.ssh/known_hosts

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants