mercurial - Hg sub-repository dependencies

Question

There have been a couple of questions about Hg sub-repo dependencies in the past (here and here) but the accepted answers don't seem to address the problem for me.

A project of mine has 4 dependencies: A, B, C, D. D is dependent on A, B and C; and B and C are dependent on A:

I want to use Hg sub-repositories to store them so I can track what version of each they rely on. This is because, while I am using A,B,C and D in this project, other projects will require just A and B. Therefore B and C must track what version of A they need independently of D. At the same time, in my application the versions of B and C referenced by a given version of D must always use the same version of A as that referenced by the given version of D (otherwise it will just fall over at runtime). What I really want is to allow them to reference each other as siblings in the same directory - i.e. D's .hgsub would look like the following, and B and C's would look like the first line.

..A = https:(central kiln repo)A
..B = https:(central kiln repo)B
..C = https:(central kiln repo)C

However this doesn't seem to work: I can see why (it'd be easy to give people enough rope to hang themselves with) but its a shame as I think its the neatest solution to my dependencies. I've read a few suggested solutions which I'll quickly outline and why they don't work for me:

1. Include copies as nested sub-directories, reference these as Hg sub-repositories. This yields the following directory structure (I've removed the primary copies of A, B, C, BA, CA as I can accept referencing the copies inside D instead):

   • project (all main project files)
   • projectD
   • projectDA
   • projectDB
   • projectDBA
   • projectDC
   • projectDCA

   Problems with this approach:

   • I now have 3 copies of A on disk, all of which could have independent modifications which must be synced and merged before pushing to a central repo.
   • I have to use other mechanisms to ensure that B, C and D are referencing the same version of A (e.g. D could use v1 while DB could use v2)

2. A variation: use the above but specify the RHS of the .hgsub to point to a copy in the parent copy (i.e. B and C should have the .hgsub below):

   A = ..A
   

   Problems with this approach:

   • I still have three copies on disk
   • The first time I clone B or C it will attempt to recursively pull the referenced version of A from "..A", which may not exist, presumably causing an error. If it doesn't exist it gives no clue as to where the repo should be found.
   • When I do a recursive push of changes, the changes in DBA do not go into the shared central repo; they just get pushed to DA instead. So if I push twice in a row I can guarantee that all changes will have propagated correctly, but this is quite a fudge.
   • Similarly if I do a (manual) recursive pull, I have to get the order right to get the latest changes (i.e. pull DA before I pull DBA)

3. Use symlinks to point folder DBA to DA etc.

   Problems with this approach:

   • symlinks cannot be encoded in the Hg repo itself so every time a team member clones the repo, they have to manually/with a script re-create the symlinks. This may be acceptable but I'd prefer a better solution. Also (personal preference) I find symlinks highly unintuitive.

Are these the best available solutions? Is there a good reason why my initial .hgsub (see top) is a pipe-dream, or is there a way I can request/implement this change?

UPDATED to better explain the wider usage of A,B,C,D

Answer 1

Instead of trying to manage your dependencies via Mercurial (or with any SCM for that matter), try using a dependency management tool instead, such as Apache Ivy.

Using an Ivy based approach, you don't have any sub-repos, you would just have projects A, B, C and D. A produces an artifact (e.g. a .jar, .so or .dll, etc), which is published into an artifact repository (basically a place where you keep your build artefacts) with a version. Projects B and C can then depend on a specific version of A (controlled via a ivy.xml file in each project) which Ivy will retrieve from the artifact repository. Projects B and C also produce artefacts that are published to your repository. Project D depends on B and C and Ivy can be told to retrieve the dependencies transitively, which means it will get the artifacts for B, C and A (because they depend on A).

A similar approach can be used with Apache Maven and Gradle (the later uses Ivy)

The main advantages are that:

• it makes it very clear what versions of each component a project is using (sometimes people forget to check .hgsub, so they don't know they are working with subrepos),
• it makes it impossible to change a dependant project (as you are working with artifacts, not code)
• and it saves you from having to rebuild dependent projects and being unsure of what version you are using.
• saves you from having multiple redundant copies of projects that are used by other projects.

EDIT: Similar answer with a slightly different spin at Best Practices for Project Feature Sub-Modules with Mercurial and Eclipse?