Add Foundry-Local as agent #410

Activity

1. Dongbo Wang (daxian-dbw)

   closed this as completedin #413on Aug 28, 2025
2. daxian-dbw commented on Aug 28, 2025

   

   Dongbo Wang (daxian-dbw)

   on Aug 28, 2025

   Member

   More actions

   With #413 being merged, you should be able to use the openai-gpt agent against a Foundry Local service endpoint.

   Use Foundry Local with openai-gpt agent

   1. Update the openai.agent.json file with the following GPT configuration:

      {
        "GPTs": [
          {
            "Name": "foundry-local",
            "Description": "A GPT instance using Foundry Local.",
            "Endpoint": "http://127.0.0.1:56952/v1",
            "ModelName": "Phi-3.5-mini-instruct-generic-cpu",
            "Key": "OPENAI_API_KEY"
          }
        ]
      
        "Active": "foundry-local"
      }

   2. In PowerShell, run the following commands to start up the Foundry service and load the model (use phi-3.5-mini as an example)

      PS> foundry service start
      🟢 Service is already running on http://127.0.0.1:56952/.
      
      PS> foundry model load phi-3.5-mini
      🕔 Loading model...
      🟢 Model phi-3.5-mini loaded successfully
      
      PS> foundry service ps
      Models running in service:
           Alias                          Model ID
      🟢  phi-3.5-mini                   Phi-3.5-mini-instruct-generic-cpu
      

   Be noted that:

   1. The endpoint to use is the URL shown from foundry service start plus /v1.
   2. The model name to use is shown as the 'Model ID' from the foundry service ps command.
      • Make sure you use the exact casing as shown in 'Model ID'. Foundry Local is case-sensitive.
   3. The API key is hardcoded to be literally "OPENAI_API_KEY".

   ❤️React with ❤️1Reacted by Alexander Buhmann