sdlc.dev

Python path · Memory and modules · Topic 8 of 12

Modules, imports and packaging

Not started

import x is a statement that runs code. The first time a module is imported, Python finds the file, compiles it, executes it top to bottom in a fresh module namespace, and stores the resulting module object in sys.modules. Every later import x anywhere in the program gets that same object back. Packaging tools, virtual environments and pyproject.toml all exist to control which files that search finds.

Why it matters: "It works on my machine" in Python is almost always an import path or an environment problem; knowing the mechanics turns it into a five-minute fix.

  1. 1A module runs once

    The first import executes the file top to bottom and caches the module object in sys.modules; later imports just look it up.

  2. 2sys.path decides

    Python searches a list of directories in order, starting with the script's own directory, so a local file named like a library can shadow it.

  3. 3One environment per project

    A virtual environment is a separate site-packages directory with its own pip, so projects don't fight over versions.

  4. 4pyproject.toml describes the project

    Its [project] table lists the name, version and dependencies, and [build-system] says which tool builds the package.

1

How it works

The explanation, with examples you can run.

What import does

import sysimport json print("json" in sys.modules)   # Trueimport json as againprint(again is json)           # True: the second import is a dict lookupprint(json.__name__, __name__) # json __main__

Because a module body runs only once, module-level code acts as one-time setup, and module-level variables act as process-wide singletons. The file you run directly is named "__main__" rather than its file name, which is what the if __name__ == "__main__": guard tests: that code runs when the file is executed as a script but not when it is imported.

A package is a directory of modules, normally with an __init__.py that runs when the package is first imported. from . import models is a relative import inside a package; it only works when the file is imported as part of that package (for example with python -m mypkg.cli), not when run as a loose script.

Imports run in order, and a module is registered in sys.modules before its body finishes. That is why circular imports fail in a confusing way: if a imports b and b immediately uses a.X, b gets the half-initialised a and raises AttributeError: partially initialized module 'a' has no attribute 'X' (most likely due to a circular import).

Where Python looks

The search goes through sys.path in order: the directory of the script (or the current directory for -m and the REPL), then entries from PYTHONPATH, then the standard library, then site-packages, where installed packages live. First match wins. A file called random.py or json.py in your project shadows the standard module, a classic source of baffling errors.

Virtual environments

Each Python installation has one site-packages. A virtual environment is a lightweight directory with its own site-packages and a python that uses it, while sharing the base interpreter's standard library:

python3 -m venv .venv            # create it (once per project)source .venv/bin/activate        # put .venv/bin first on PATH (Windows: .venv\Scripts\activate)python -m pip install requests   # installs into .venv only

Activation only changes PATH; running .venv/bin/python directly works just as well. Using python -m pip rather than a bare pip guarantees that pip installs into the interpreter you are actually going to run. Many Linux distributions and Homebrew now mark their system Python as externally managed (PEP 668), so pip refuses to install into it outside a venv.

pyproject.toml

Modern projects describe themselves in pyproject.toml. The [project] table (PEP 621) holds the name, version, requires-python and dependencies; [build-system] (PEP 517 and 518) names the backend, such as setuptools, hatchling or flit-core, that turns the source into a wheel, the pre-built archive pip installs. pip install . reads this file, builds a wheel and installs it. Dependency lists in pyproject.toml say what a package can work with; to pin exact versions for an application, use a lock file. Tools such as pip-tools, Poetry, PDM and uv produce them, and PEP 751 defines a standard pylock.toml format.

2

Check yourself

3 questions. Get them all right to finish this topic.

  1. 1.Two modules import each other at the top, and one uses an attribute of the other at import time. What usually happens?
  2. 2.You create a file called `json.py` next to your script. What does `import json` in the script now load?
  3. 3.On a recent macOS (Homebrew) or Debian/Ubuntu system Python, `pip install requests` fails with "externally-managed-environment". What is the intended fix?

0 of 3 right so far. Get all three to finish this topic.

Go deeper