Refactor the dispatching functions

[YuanmingLeee]

Dispatch with model ID to a machine learning online service with a CLI.

• The model will first be downloaded or already downloaded by model hub manager
• The model will be dispatched to as a service by its specific serving engine
• Current support engine dispatch with CLI: TorchScript, ONNX, TFS and TensorRT(optional)
• The dispatch device (CPU, CUDA:0) should be specified with CLI options
• The CLI also support for distributed dispatch later, leave the IP as localhost for extension.
• Use typer to implement
• add unit test

[QichenZheng]

I'm not sure what typer refers to, could you give me a more specific explanation？

[YuanmingLeee]

Refer to this package:
https://typer.tiangolo.com

[QichenZheng]

When I tried to execute “sh scripts/start_service.sh” according to CONTRIBUTING.md to Install Service, I found that there was no file named "start_service.sh" in the scripts file.

[HuaizhengZhang]

@univerone Could you help update the CONTRIBUTING.MD and README.MD according to the recent merged PRs?

[QichenZheng]

When I try to install model CI in a new enviroment with python 3.7.1 ,I meet following problem:

Installing Conda environment...OK
Installing Triton client API...OK
Generating gRPC code.../home/zqc/.conda/envs/modelci/lib/python3.7/runpy.py:125: RuntimeWarning: 'grpc_tools.protoc' found in sys.modules after import of package 'grpc_tools', but prior to execution of 'grpc_tools.protoc'; this may result in unpredictable behaviour
warn(RuntimeWarning(msg))
(modelci1)

I have tried all the possible reason that may cause failure of installation but still cant't moving forward.
PS:
Software and Hardware Versions
Ubuntu 18.04.6 LTS
CUDA 10.2
Docker version 19.03.6

[YuanmingLeee]

When I try to install model CI in a new enviroment with python 3.7.1 ,I meet following problem:

Installing Conda environment...OK
Installing Triton client API...OK
Generating gRPC code.../home/zqc/.conda/envs/modelci/lib/python3.7/runpy.py:125: RuntimeWarning: 'grpc_tools.protoc' found in sys.modules after import of package 'grpc_tools', but prior to execution of 'grpc_tools.protoc'; this may result in unpredictable behaviour
warn(RuntimeWarning(msg))
(modelci1)

I have tried all the possible reason that may cause failure of installation but still cant't moving forward.
PS:
Software and Hardware Versions
Ubuntu 18.04.6 LTS
CUDA 10.2
Docker version 19.03.6

Have you try start modelci service? In my point of view, a warning does not indicate the installation fails. You can ignore them.

[QichenZheng]

I am confused about the specific version requirements and dependencies of each package. This is an error that occurred when I started the service according to the contribution.md, which seems to be related with 'protobuf'.

（Note: This environment is the new environment I configured this morning and satisfied the requirements of the environment in contributing.md）
Software and Hardware Versions
Ubuntu 18.04.6 LTS
CUDA 10.2
Docker version 19.03.6
anaconda 4.9.2

(modelci) zqc@zqc-Lenovo-Y520-15IKBN:~/ML-Model-CI$ python modelci/cli/init.py service init
Traceback (most recent call last):
File "modelci/cli/init.py", line 18, in
from modelci.cli import modelhub
File "/home/zqc/ML-Model-CI/modelci/init.py", line 9, in
from . import hub
File "/home/zqc/ML-Model-CI/modelci/hub/init.py", line 10, in
from .profiler import Profiler
File "/home/zqc/ML-Model-CI/modelci/hub/profiler.py", line 13, in
from modelci.hub.client import CVONNXClient
File "/home/zqc/ML-Model-CI/modelci/hub/client/init.py", line 8, in
from .onnx_client import CVONNXClient
File "/home/zqc/ML-Model-CI/modelci/hub/client/onnx_client.py", line 18, in
from modelci.types.proto.service_pb2 import InferRequest
File "/home/zqc/ML-Model-CI/modelci/types/proto/service_pb2.py", line 21, in
create_key=_descriptor._internal_create_key,
AttributeError: module 'google.protobuf.descriptor' has no attribute '_internal_create_key'

Before that, I strictly followed the steps of contributing.md to install the required environment.
libprotobuf=3.8.0 and protobuf=3.8.0 are also explicitly required in enviroment.yml

In addition, the version requirements of tensorflow and scipy in environment.yml conflict
tensorflow 2.1.0 (writed in the enviroment.yml) requires scipy==1.4.1; python_version >= "3", but you'll have scipy 1.5.2 (writed in the enviroment.yml) which is incompatible.

There are also contradictions between the requirements in enviroment.yml and requirements.txt
pytorch-lightning 1.1.4 (required in requirements.txt) requires tensorboard>=2.2.0, however tensorflow 2.1.0 requires tensorboard<2.2.0,>=2.1.0

[QichenZheng]

When I try to install model CI in a new enviroment with python 3.7.1 ,I meet following problem:

Installing Conda environment...OK
Installing Triton client API...OK
Generating gRPC code.../home/zqc/.conda/envs/modelci/lib/python3.7/runpy.py:125: RuntimeWarning: 'grpc_tools.protoc' found in sys.modules after import of package 'grpc_tools', but prior to execution of 'grpc_tools.protoc'; this may result in unpredictable behaviour
warn(RuntimeWarning(msg))
(modelci1)

I have tried all the possible reason that may cause failure of installation but still cant't moving forward.
PS:
Software and Hardware Versions
Ubuntu 18.04.6 LTS
CUDA 10.2
Docker version 19.03.6

Have you try start modelci service? In my point of view, a warning does not indicate the installation fails. You can ignore them.

I create a new enviroment and install the grpc_tools.The installation of modelci was successful, but this error occurred when I start the modelci service

(modelci2) zqc@zqc-Lenovo-Y520-15IKBN:~/ML-Model-CI$ modelci service init
2021-03-17 10:02:41.941063: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
Traceback (most recent call last):
File "/home/zqc/.conda/envs/modelci2/bin/modelci", line 5, in
from modelci.cli import cli
ImportError: cannot import name 'cli' from 'modelci.cli' (/home/zqc/.conda/envs/modelci2/lib/python3.7/site-packages/modelci/cli/init.py)

[YuanmingLeee]

You should go through CONTRIBUTING.md.

As we are rapidly developing, the env in requirement.txt may out of date. So when you do pip install, you have the conflict issues. Follow instructions in CONTRIBUTE.md. Open an issue if you encounter a problem then.

[univerone]

@univerone Could you help update the CONTRIBUTING.MD and README.MD according to the recent merged PRs?

OK!

[QichenZheng]

You should go through CONTRIBUTING.md.

As we are rapidly developing, the env in requirement.txt may out of date. So when you do pip install, you have the conflict issues. Follow instructions in CONTRIBUTE.md. Open an issue if you encounter a problem then.

I have always followed the steps of CONTRIBUTING.md.Only refer to requirements.txt on the version of pytorch-lightning, The versions of other packages are the same as environment.yml. Regarding the problem I encountered, I have created a new issue #252 #

[YuanmingLeee]

You should go through CONTRIBUTING.md.
As we are rapidly developing, the env in requirement.txt may out of date. So when you do pip install, you have the conflict issues. Follow instructions in CONTRIBUTE.md. Open an issue if you encounter a problem then.

I have always followed the steps of CONTRIBUTING.md.Only refer to requirements.txt on the version of pytorch-lightning, The versions of other packages are the same as environment.yml. Regarding the problem I encountered, I have created a new issue #252 #

Actually, you don't need the pytorch lightning for this function. There may be some version conflicts between pytorch lightning and protobuf. You can try StackOverflow for the solution, feel free to comment on #252 if you can't get started.