Run Poetry commands from Python

[JnsLns]

Hello,

is there a simple way to run Poetry commands from within a Python program? By that I mean pure Python, without making the detour via a subprocess that runs poetry <rest of command>.

My use case: I am building a machine learning model in a poetry environment and then upload the model artifact to a model registry (MLflow). I would like to bundle the dependency specifications with the model artifact, so that the environment to run the model can be recreated elsewhere later. I would like to include it in various non-poetry fomats (at least requirements.txt, ideally conda.yaml). Since I am training and logging the model within one Python script, it would make sense if I could do what poetry export does from that script as well.

Looking through the poetry package I clawed my way to something like this:

from poetry.poetry import Poetry
from poetry_plugin_export.exporter import Exporter
import pathlib

exporter = Exporter(
    poetry = Poetry(<.....>),
    io = <.....>
)
exporter.export(
    fmt = 'requirements.txt',
    cwd = pathlib.Path('/where/we/want/the/file'),
    output = 'requirements.txt'
)

I feel it can be done this way, I am just figuring out how I need to instantiate the Poetry object (ellipses indicate where I am still struggling).

Yet, it seems overly laborious. I feel I am missing the obvious, standard way of doing this?

Thanks!

[Secrus]

We don't provide any stable API to interact with Poetry as a library. There is a plan to introduce a stable plugin API, which would be close to what you are asking for, but it has no real ETA.

[JnsLns]

Thanks for the info! Good to know at least that I am not missing the easy way. Probably better to go the subprocess route then, given that (I suppose) the package API may be subject to breaking changes.

[nadworny]

hi @JnsLns , did you manage to find a clean way to do it?

[JnsLns]

@nadworny Not a clean one in the sense that it would stay within Python. I simply went the subprocess route.

import subprocess

def get_env_requirements() -> str:
    '''Run `poetry export` to get content of requirements.txt as string '''    

    subproc_out = subprocess.run(        
        'poetry export',
        shell=True,
        capture_output=True,
        encoding='utf-8'
    )

    if subproc_out.returncode != 0:
        raise RuntimeError('Unable to export requirements')

    return subproc_out.stdout   

[mansenfranzen]

For debugging purposes regarding the development of poetry plugins, I also required to invoke poetry from plain python. The following worked for me:

from poetry.console.application import Application
from cleo.io.inputs.argv_input import ArgvInput
import sys


input = ArgvInput(["poetry", "build"])
input.set_stream(sys.stdin)

Application().run(input=input)

This is basically the same as invoking from cmd via poetry build.