I am definitely impressed with the new capabilities announced on Mlflow 2.6.0 around an API Gateway , i have not tested yet.

Using this article to figure ou the main capabilities of it, as far as i can see i can see the ability to :

  1. Abstract multiple LLMs API’s

  2. Provide some Governance

  3. Fastrack the creation and deployment of LLM projects

On the next notes i will start playing with it.