version control - Projects within projects using Git

Question

How do I set up a Git project to contains other projects?

eg. I am working on an online mapping app. We developed a GPS tool together with an outfit in SF. We simultaneously developed a Python Geomapping script together with a different concern (that only cares about geomapping). Our own core files unite the two, and build upon them for the app we need.

Each of the projects must exist by itself - the folks that have interest in the GPS only have interest in GPS - but the "parent" project which includes all of the others must be accessible as a project.

I've spent some time trying to understand submodules, but they appear to have too much independence for what is needed.

Also, if possible, it would be nice if each of those projects could contain one or two overlapping scripts. Could one Git project include a file that is not part of its 'root' so that when this file is updated by either team both can benefit?

Is this doable with Git? With Mercurial? Does the host (GitHub, Gitorious) matter?

I have the idea of using Subversion for the 'parent' - ignoring the .git folders, and using Git for the projects (ignoring .svn folders) - but that is only a last resort.

edit:

To explain why I don't want Submodules:

1. When users download, the zip does not include the submodules (here & here). Ditto when even collaborators try to setup the project. This is a show stopper.
2. Submodules are frozen - they do not (easily) pick up the latest version of the project that is being pointed to.
3. Other reasons as pointed out in the fantastic answers below and in this monologue at NoPugs.

Subtree-merging (introduced to me by Paul, below) will not do: It is difficult to update the source [of a subtree] from within the project it is merged into, and that source must reside outside of the 'root' folder of the project. Being a web app, it is vital that all my pages link internally to a folder within them, and that testing and updates be done directly within that folder. (Hope this is clear and useful to others.)

Still studying setting up 'remote branches' but other ideas are still welcome.

Answer 1

I haven't found submodules to be particularly useful on the (small) projects I've worked on. Once you've set them up, working on the whole project requires adding additional parameters to almost every command and the syntax isn't completely regular. I imagine if I worked on larger projects with more submodules, I'd see it as a more beneficial tradeoff.

There are two possibilities that keep the sub-projects as independent git repos that you pull from into your main (integration) repo:

• Using subtree merge to bring your external projects into separate subdirectories in your main repo that includes your core files. This makes it easy to update the main project from the external projects, but complicated to send changes back to the external projects. I think of this as a good way to include project dependencies, but it wouldn't work so well with shared files. Another simple explanation (link fixed).

• Set up each project as a remote branch in your main repo and merge from each of them into your master (integration) branch that also contains your core files. This requires some discipline: if you make any changes to the external projects in your main repo, they must be made in the branch and then merged into the master; and you never want to merge into the project branches. This makes it easy to send changes back to the external projects and is a perfectly acceptable use of branches in Git.

  Your shared scripts can be handled as another independent branch in your main directory which your external partners can pull from and push to as a remote branch.

If you try to run SVN & Git in the same directory, you make it very hard to use branching in either system, because SVN does branching by copying file directories while Git tracks pointers. Neither system would automatically see branches you make in the other. I think that 'solution' is more trouble than it is worth.