Unlocked Package with Dependency in a different folder

Question

In the past I was able to have a base package with its repo in one folder and then in a completely separate folder the app package with its repo. The sfdx force:source:push command would push the dependent code and the app code if it wasn't already there. However, lately it seems something has changed since this is no longer the case for me. I started getting Unexpected file found in package directory: ..\Source Code\Salesforce\Base_Package\.git\FETCH_HEAD errors when trying to create a package version with the Salesforce CLI.

The push command seems to ignore the dependent directory listed in the sfdx-project.json file and the version create command gives the above error. The only workaround I found to get the push command to work, is if I move the Base_Package folder inside of the App_Package folder. I haven't tried with the version create command yet. Doing this is unsustainable since I have multiple App_Packages in different folders since each one has its own repo that are dependent on the Base_Package.

Config (Not working) - I know it is finding the folder because I do not get an invalid package directory error.
Folder Structure

• Base Package: ../Source Code/Salesforce/Base_Package
• App Package: ../Source Code/Salesforce/salesforce-core/app_package

sfdx-project.json file contents

{
  "packageDirectories": [
      {
        "path": "../../Base_Package",
        "default": false,
        "package": "Base_Package",
        "versionName": "ver 1.0.XX",
        "versionNumber": "1.0.XX.NEXT"
    },
    {
      "path": "force-app",
      "default": true,
      "package": "App Package",
      "versionName": "ver 0.0.1",
      "versionNumber": "0.0.1.NEXT",
      "dependencies": [
        {
          "package": "Base_Package",
          "versionNumber": "1.0.XX.LATEST"
        }
      ]
    }
  ],
  "pushPackageDirectoriesSequentially": true,
  "name": "app-package",
  "namespace": "",
  "sfdcLoginUrl": "https://login.salesforce.com",
  "sourceApiVersion": "54.0",
  "packageAliases": {
    "Base_Package": "0H...",
    "Base_Package@1.0.XX-1": "04t...",
    "Base_Package@1.0.XX-2": "04t...",
    "Base_Package@1.0.XX-3": "04t..."
  }
}

Config (Works but not feasable to move the Base_Package to each App_Package folder for development or package version creation.)
Folder Structure

• Base Package: ../Source Code/Salesforce/Salesforce-core/app_package/Base_Package
• App Package: ../Source Code/Salesforce/salesforce-core/app_package

sfdx-project.json file contents

{
  "packageDirectories": [
      {
        "path": "./Base_Package",
        "default": false,
        "package": "Base_Package",
        "versionName": "ver 1.0.XX",
        "versionNumber": "1.0.XX.NEXT"
    },
    {
      "path": "force-app",
      "default": true,
      "package": "App Package",
      "versionName": "ver 0.0.1",
      "versionNumber": "0.0.1.NEXT",
      "dependencies": [
        {
          "package": "Base_Package",
          "versionNumber": "1.0.XX.LATEST"
        }
      ]
    }
  ],
  "pushPackageDirectoriesSequentially": true,
  "name": "app-package",
  "namespace": "",
  "sfdcLoginUrl": "https://login.salesforce.com",
  "sourceApiVersion": "54.0",
  "packageAliases": {
    "Base_Package": "0H...",
    "Base_Package@1.0.XX-1": "04t...",
    "Base_Package@1.0.XX-2": "04t...",
    "Base_Package@1.0.XX-3": "04t..."
  }
}

Also tested the above with the easy-spaces-lwc repo since they have multiple folders with code and the same issue occurs when I move one of the folders outside of the main folder and redirect the sfdx-project.json folder path to the new location.

Any ideas on how to make it work like it did at first, so the Base_Package could be located in a common folder that the extension packages could reference?

Dev Environment:
Windows 10
VS Code v1.66.2
sfdx-cli/7.146.1 win32-x64 node-v16.14.2
Salesforce Extension Pack v54.8.0
sfdx plugins --core result:
@oclif/help 1.0.1
@oclif/plugin-autocomplete 0.3.0 (core)
@oclif/plugin-commands 1.3.0 (core)
@oclif/plugin-help 3.3.1 (core)
@oclif/plugin-not-found 1.2.6 (core)
@oclif/plugin-plugins 1.10.11 (core)
@oclif/plugin-update 1.5.0 (core)
@oclif/plugin-warn-if-update-available 1.7.3 (core)
@oclif/plugin-which 1.0.4 (core)
@salesforce/lwc-dev-mobile 1.5.1
@salesforce/lwc-dev-server 2.11.0
@salesforce/sfdx-plugin-lwc-test 0.1.7 (core)
alias 2.0.0 (core)
apex 0.11.0 (core)
auth 1.8.2 (core)
community 1.1.4 (core)
config 1.3.30 (core)
custom-metadata 1.0.12 (core)
data 0.6.12 (core)
generator 1.2.2 (core)
info 2.0.0 (core)
limits 2.0.0 (core)
org 1.11.2 (core)
salesforce-alm 54.1.0 (core)
schema 2.0.0 (core)
sfdx-cli 7.146.1 (core)
source 1.9.3 (core)
telemetry 1.4.0 (core)
templates 54.3.0 (core)
trust 1.1.0 (core)
user 1.7.1 (core)

Update - Visual Folder Structure

Source Code/
  |__ Salesforce/
      |__ Base_Package/
      |   |__ .git/
      |   |__ .sfdx/
      |   |__ data/
      |   |__ force-app
      |   |__ .forceignore
      |   |__ .gitignore
      |   |__ README.md
      |__ salesforce-core/
          |__ App_Package
              |__ .git/
              |__ .sfdx/
              |__ .vscode/
              |__ config/
              |__ force-app/
              |__ scripts/
              |__ .forceignore
              |__ .gitignore
              |__ package.json
              |__ package-lock.json
              |__ README.md
              |__ sfdx-project.json

Answer 1

Given your structure the sfdx-project.json must include some "../" in the path to the base package (you show "./").

This structuring is fragile, due to the in-file-system-only folder-based structural relationship across git repositories - it relies on the developer to carefully create the base folder structure and place git repo clones in the right place.

I am surprised it previously worked, to be honest (though, as per the comments, I am not clear why using a .forceignore entry of **/.git/** doesn't work).

Salesforce recommends what they call a "monorepo" for this (read more here); a single SFDX project, in one git repository, that includes folders for multiple packages (here your base package and your app package).

You would restructure your code into a single git repository, thus:

project
  |__ .git/
  |__ .sfdx/
  |__ .vscode/
  |__ config/
  |__ force-base/
  |     |__ ...
  |__ force-app/
  |     |__ ...
  |__ scripts/
  |__ .forceignore
  |__ .gitignore
  |__ package.json
  |__ package-lock.json
  |__ README.md
  |__ sfdx-project.json

The sfdx-project.json would be similar to yours:

{
  "packageDirectories": [
      {
        "path": "force-base",
        "default": false,
        "package": "Base_Package",
        "versionName": "ver 1.0.XX",
        "versionNumber": "1.0.XX.NEXT"
    },
    {
      "path": "force-app",
      "default": true,
      "package": "App Package",
      "versionName": "ver 0.0.1",
      "versionNumber": "0.0.1.NEXT",
      "dependencies": [
        {
          "package": "Base_Package",
          "versionNumber": "1.0.XX.LATEST"
        }
      ]
    }
  ],
  "pushPackageDirectoriesSequentially": true,
  "name": "app-package",
  "namespace": "",
  "sfdcLoginUrl": "https://login.salesforce.com",
  "sourceApiVersion": "54.0",
  "packageAliases": {
    "Base_Package": "0H...",
    "Base_Package@1.0.XX-1": "04t...",
    "Base_Package@1.0.XX-2": "04t...",
    "Base_Package@1.0.XX-3": "04t..."
  }
}

If you definitely don't want to restructure to follow Salesforce's structuring you have two options:

1. See if you can work out an appropriate .forceignore entry (maybe ../**/.git/**?).
2. Break the double-package-directory push and instead actually install the dependency (Base Package) as part of your scratch org creation, using scripting, instead of pushing its content. Of course, this can cause namespace issues and may not be achievable in all cases.