Git submodules as a package manager: AI's dependency problem in miniature

The most interesting software-engineering argument of the week is hiding in a blog post about a git worktree remove refusal. In Git Submodules as a Package Manager, published September 1, the author makes a precise claim: .gitmodules is a manifest, the gitlink is a lockfile entry, and the whole system behaves like a package manager that shipped without a resolver, a shared cache, or version ranges. For the AI industry, that is not a historical curiosity. It is the shape of the dependency problem that model weights, training pipelines, and agent toolchains are about to hit head-on.

The post opens with a concrete collision. The author added a worktree to a repository, ran git submodule update --init to populate vendored dependencies, and then found that git worktree remove refused to clean up. Git’s man page says unclean worktrees or ones with submodules need --force. Submodules get their own clause, distinct from dirty state. git worktree move refuses outright. GitHub’s git 2.5 announcement introduced worktrees in July 2015 with a one-line caveat: “It’s not recommended to use git worktree with a repository that contains submodules.” Eleven years later, the friction persists.

The deeper point is structural. The gitlink, a commit SHA recorded at a path with mode 160000, is the lockfile entry. The .gitmodules file, mapping paths to fetch URLs, is the manifest. git submodule update reads both and populates the working tree. That is the install step. The pin is as precise as any package manager’s: an exact commit identified by object ID.

Then the abstraction breaks. The gitlink records only which commit to check out, so .gitmodules carries a URL per submodule and update clones from there. That is the only resolution mechanism. If the upstream repository is renamed, transferred to a different host, or taken private, every downstream pin breaks, even though the SHA is unchanged and the objects still exist in every clone that already has them. Git has no lookup from a commit ID to servers that hold it. The manifest hard-codes a host.

That is the AI problem in miniature. Model weights are increasingly distributed as Git repositories, often via Git LFS or dedicated weight registries. Hugging Face, the dominant hub, hosts tens of thousands of model repos. But the dependency graph of a serious AI project is not just one model. It is a base model, a fine-tune, a tokenizer config, a set of evaluation harnesses, a vector database schema, a prompt template pack. Each of those can be a repository. Each can be pinned at a commit. And none of them, under the gitlink model, can be resolved by identity alone.

The post catalogues the failure modes with the specificity of someone who has lived them. Storage is the most visible. A submodule’s git directory lives under the superproject’s $GIT_DIR/modules/<name>/, with a .git file in the submodule’s working tree containing a gitdir: pointer back to it. Each entry under modules/ is a git directory with its own refs, HEAD, index, config, hooks, and by default its own object store. Removing a submodule is spread across three places: git rm <path> drops the gitlink and the .gitmodules entry, git submodule deinit <path> clears the working tree and the .git/config entry, and the absorbed $GIT_DIR/modules/<name> directory that both leave behind is documented as a manual rm -rf.

The multiplication gets worse when two submodules both depend on a third repository. Each path in the superproject gets its own modules/ entry, its own object store unless alternates are configured by hand, and its own gitlink. The two pins can point at different commits of the same repository, and git treats them as unrelated checkouts. Package managers with a shared cache, cargo’s registry cache, pnpm’s content-addressable store, the Go module cache, store the bytes once and check them out per location. Git stores them once per path. For a model repo that is 10 gigabytes, that is not a footnote. That is a disk bill.

The Rust project’s experience, cited in the post, is the cautionary tale. Their account of moving compiler subprojects off submodules lists the cluster from experience: checkouts left empty or on the wrong commit after clone, unrelated submodule bumps landing in pull requests because a branch switch left the submodule dirty, and custom logic in the bootstrap build tool to check each submodule out to the right commit before building. The Rust compiler moved off submodules years ago. AI projects are now walking into the same trap, often with larger payloads and less experienced infrastructure teams.

Updating is where the model-pinning analogy gets sharpest. The gitlink holds one commit SHA, so moving a submodule forward means entering it, fetching, checking out the new commit, leaving, and git add <path> in the superproject to record the new gitlink. git submodule update --remote fetches the configured branch’s tip and checks that out instead of the recorded gitlink. .gitmodules can name a branch per submodule for --remote and the update bots to follow. There is no syntax for a version range, a tag pattern, or a minimum commit. The manifest’s only floating reference is a branch name, and the gitlink is the only pin.

For AI, that is a governance hole. A model card says “fine-tuned from Llama 3.1 8B at commit abc1234.” The pin is exact. But the training data, the tokenizer, the evaluation suite, and the inference runtime are each separate repos with separate gitlinks. When the evaluation suite updates its prompt format, does the model card change? When the tokenizer repo adds a new merge, does every downstream fine-tune silently shift? Branch-tip following via Dependabot and Renovate, both of which the post notes can open pull requests bumping a gitlink, makes the drift automatic. Dependabot’s gitsubmodule ecosystem proposes a new gitlink SHA when the submodule’s configured branch moves. Renovate’s git-submodules manager does the same, shipping disabled by default. Both follow branch tips because a branch name is the only reference the manifest exposes.

Security is the part AI builders cannot ignore. .gitmodules is committed to the repository, so a hostile upstream controls its contents, and git parses it during clone --recurse-submodules before the user has seen any of the fetched files. That combination has produced remote code execution repeatedly. CVE-2018-11235 used ../ in a submodule’s name so its git directory, hooks included, was written outside $GIT_DIR/modules/ and a post-checkout hook ran during clone. CVE-2018-17456 had the submodule URL begin with -, so the child git clone parsed it as an option. CVE-2022-39253 was a disclosure bug: a symlink in a submodule’s object directory made a local-transport clone copy arbitrary files from the victim’s disk. CVE-2024-32002 combined a symlink with a case-insensitive filesystem to write a hook into .git/ during recursive clone.

The AI supply chain is a target-rich environment for exactly this class of attack. Model repos are fetched by CI systems, training clusters, and evaluation harnesses, often with elevated credentials. A poisoned .gitmodules in a popular fine-tune repo is a foothold into every downstream pipeline that clones it recursively. The fix for CVE-2022-39253 changed the protocol.file.allow default to user, so local-path submodules now need an explicit opt-in. That is a band-aid on a design where the manifest is executable before inspection.

The post’s closing observation is the one worth taking into the AI infrastructure conversation. Submodules expose git’s internals directly: object IDs as the pin, detached HEADs after update, the $GIT_DIR/modules/ layout, transport URLs in the manifest. A package manager wraps the equivalents behind a manifest format, a resolver, and a local cache. Submodules surface them raw. Most of the gaps map to things package managers already solved: a shared object cache, recursing into dependencies by default on clone and checkout, a single lifecycle for adding and removing a dependency, range constraints in the manifest.

The April patch series adding --recurse-submodules to git worktree add, mentioned in the post, tackles one instance of the storage problem, giving each worktree its own submodule checkout over hardlinked shared storage. Resolution is the harder one. A commit SHA is a host-independent identity for the object, and the URL in .gitmodules is git’s only mapping from that identity to a server that holds it.

For AI builders, the lesson is not to abandon Git. It is to stop pretending that a gitlink is a dependency specification. When a model card pins a base model by SHA but the training harness follows a branch tip, the reproducibility promise is fiction. When a 10-gigabyte weight repo is cloned once per dependent path, the storage bill is the tax on that fiction. The Rust project paid it, documented it, and left. AI projects, with larger artifacts and tighter supply-chain scrutiny, will pay it faster. The question is whether the industry builds a real resolver for model and dataset dependencies before the first high-profile poisoning incident forces the issue. Git submodules are a package manager the way a hammer is a nail gun. Both drive pins. One of them leaves the user holding the handle.