How to set a source for a sub dependency of a direct dependant package?

[wwuck]

To make this example easier, I have two packages: package-a and package-b. Both generated as simple packages with poetry new package-a and poetry new package-b.

package-a:

[tool.poetry]
name = "package-a"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.11"


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

package-b:

[tool.poetry]
name = "package-b"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.11"


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

I then add a new dependency into package-b for the torch package with a custom source set:

[tool.poetry]
name = "package-b"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.11"
torch = [
  {source = 'torchcpu', version = '2.3.1+cpu'},
]

[[tool.poetry.source]]
name = 'torchcpu'
priority = 'explicit'
url = 'https://download.pytorch.org/whl/cpu'


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

package-b imports and uses torch directly while package-a only uses the api of package-b so it doesn't import or use torch directly.

If I build a wheel for package-b and try to add the package-b wheel as a dependency of package-a it fails:

/tmp/package-a$ poetry add ../package-b/dist/package_b-0.1.0-py3-none-any.whl 

Updating dependencies
Resolving dependencies... (0.1s)

Because package-b (0.1.0) @ file:///tmp/package-b/dist/package_b-0.1.0-py3-none-any.whl depends on torch (2.3.1+cpu) which doesn't match any versions, package-b is forbidden.
So, because package-a depends on package-b (0.1.0) @ file:///tmp/package-b/dist/package_b-0.1.0-py3-none-any.whl, version solving failed.

Is there any way for package-b to tell package-a about the source url for torch? I can't see anything being added to the package-b wheel metadata for this.

Do I need to add torch as an explicit dependency with custom source into package-a/pyproject.toml as I did for package-b even if package-a is not directly using or importing torch?

[radoering]

Is there any way for package-b to tell package-a about the source url for torch? I can't see anything being added to the package-b wheel metadata for this.

No, there is no way to do this because explicit package sources is a Poetry feature so this information cannot be added to metadata when building a wheel.

Do I need to add torch as an explicit dependency with custom source into package-a/pyproject.toml as I did for package-b even if package-a is not directly using or importing torch?

Yes, but you can add it without a specific version constraint in an optional dependency group so that it is only installed if it is required by a direct dependency. For example:

[tool.poetry]
name = "package-a"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.11"
package-a = ">=0.1.0"

[tool.poetry.group.transitive]
optional = true

[tool.poetry.group.transitive.dependencies]
torch = {source = 'torchcpu', version = '*'}

[[tool.poetry.source]]
name = 'torchcpu'
priority = 'explicit'
url = 'https://download.pytorch.org/whl/cpu'

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Alternatively, you can just add the package source as supplemental if you do not care that torch is first searched on PyPI and only on torchcpu if the version is not found on PyPI:

[tool.poetry]
name = "package-a"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.11"
package-a = ">=0.1.0"

[[tool.poetry.source]]
name = 'torchcpu'
priority = 'supplemental'
url = 'https://download.pytorch.org/whl/cpu'

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

[wwuck]

Thanks. Adding the transitive optional dependency with explicit source works here.