Why It Works Locally but Fails in CI: 4 Hidden Culprits and How to Fix Them

When a requirements.txt entry lacks an upper bound, the dependency resolver will accept any newer release that satisfies the lower constraint. In a clean CI environment this behavior becomes visible the moment a library publishes a breaking version. For example, the specification sqlmodel>=0.0.45,<0.0.48 deliberately caps the acceptable range; removing the ,<0.0.48 clause would allow sqlmodel 0.0.48 (or later) to be installed automatically. If that release introduces a transitive dependency change, the CI job can fail even though the same code runs locally on a developer’s machine that still has the older version cached.

Understanding why the failure occurs requires separating two concepts:

• Version drift: The set of packages installed in CI is built from scratch, so any unpinned range will drift to the latest matching version at build time.
• Environment parity: Local environments often retain older packages, hidden behind caches or manual upgrades, masking the incompatibility until CI reports it.

To prevent silent breaks, adopt a disciplined pinning strategy:

• Use an exact version (sqlmodel==0.0.47) for production‑critical libraries.
• When flexibility is needed, combine a lower bound with a strict upper bound (sqlmodel>=0.0.45,<0.0.48) to limit exposure to known‑good releases.
• Generate a lock file (e.g., pip-tools requirements.txt → requirements.lock, poetry.lock, or conda‑env.yml) and commit it to version control.
• Integrate a dependency‑audit step in the CI pipeline that compares the resolved versions against the lock file and fails on mismatches.
• Periodically review and update the upper bounds after a controlled test of the newer version in an isolated branch.

Practical example:

# requirements.in
sqlmodel>=0.0.45,<0.0.48
fastapi==0.104.0

# Generate lock file
pip-compile requirements.in -o requirements.txt

By committing requirements.txt and ensuring the CI job installs with pip install -r requirements.txt, the build will always use the exact versions recorded in the lock file. If a new sqlmodel release appears, the CI run will still succeed, and the change will be detected only when a developer intentionally updates the .in file and regenerates the lock. This disciplined approach eliminates the “works on my machine” symptom caused by unpinned dependencies and maintains reliable, repeatable builds across the entire development lifecycle.

When a test suite passes on a developer’s workstation but fails in continuous integration (CI), the most common cause is a hidden divergence between the two environments. CI jobs start from a clean image, installing only what the pipeline declares. Anything that exists only on a local machine—different lint tool versions, a mismatched Python interpreter, or globally‑installed packages—remains invisible until the CI run surfaces the discrepancy.

• Lint versions: A project may use flake8 5.0 locally while the CI image provides version 4.0. New rules introduced in the newer version can cause failures that do not appear locally.
• Python interpreter: Running code under Python 3.10 on a laptop but under Python 3.9 in CI can expose syntax errors (e.g., the match statement) or changes in the standard library.
• Globally installed packages: Packages installed outside the declared requirements.txt—such as pytest‑cov or a custom myutils module—are available locally but absent in CI, leading to import errors.
• OS‑level libraries: System dependencies (e.g., libxml2 or openssl) may differ between a developer’s OS and the CI container, affecting compiled extensions.

To expose these hidden differences, reproduce the CI command in a clean, isolated environment before blaming the pipeline:

1. Copy the exact command line shown in the CI log (including interpreter flags and environment variables).
2. Create a temporary directory and initialise a fresh virtual environment: python -m venv .venv && source .venv/bin/activate.
3. Install the project’s pinned dependencies exactly as CI does (e.g., pip install -r requirements.txt).
4. Run the copied command. If it fails, the issue is environmental; if it passes, the problem lies in the CI configuration.

Using containerisation (Docker) or reproducible environment tools (Poetry, Pipenv) can automate this process, ensuring the interpreter, lint tools, and package set match the CI image. By systematically aligning the local execution environment with the CI definition, engineers eliminate “works on my machine” false positives and reduce the time spent chasing elusive environment‑related failures.

When a test reads a file that is not part of the repository, the test can succeed on a developer’s workstation but fail in a clean CI environment. The root cause is that the file is either listed in .gitignore or was never added to Git, so the CI runner never receives it. This situation is one of the four common “non‑code” reasons for flaky CI builds described in community discussions.

Typical symptoms include:

• Local pytest (or equivalent) runs without error.
• CI job aborts with FileNotFoundError or a similar exception.
• The failing line points to a data file, configuration file, or fixture that exists only on the developer’s disk.

To understand why the file is missing, first verify that the test’s file access is intentional and not a side‑effect of a temporary artifact. For example, a test that loads tests/fixtures/sample.json may rely on a copy of that file that was created manually and never committed.

One‑command check

The following pipeline captures every file opened by the test suite and cross‑references it with the Git index. If any accessed path is not tracked, the command exits with a non‑zero status and prints the offending file.

strace -e openat -f -o /tmp/trace.txt pytest \
  && cut -d'"' -f2 /tmp/trace.txt | sort -u \
  | while read f; do git ls-files --error-unmatch "$f" >/dev/null || echo "Untracked: $f"; done \
  | grep -q . && exit 1 || exit 0

Explanation of the pipeline:

• strace -e openat -f records every openat system call made by the test process.
• The cut and sort steps extract a unique list of file paths.
• git ls-files --error-unmatch succeeds only for files that are versioned; otherwise it prints “Untracked”.
• The final grep determines whether any untracked files were found and sets the exit code accordingly.

Integrating this check into the CI pipeline (e.g., as a pre‑test step) guarantees that any file a test depends on is present in the repository, aligning the local development environment with the clean CI environment and eliminating the “works on my machine” failure mode.