Introduction

OpenAI gives some interesting ideas on how to use their services on the API page. Using the 'Summarization' option as an example, how can this functionality be delivered to other projects and endpoints as safely, simply and cheaply as possible?

Why Azure Function

At first, adding this extra layer between consumer and OpenAI may seem counterintuitive. It adds another service to develop, maintain and run.

Using Azure Functions as a gateway between consumer and service (openAI) has the following advantages:

Accessibility

While accessing OpenAI from almost any service is possible using the REST interface, not all client applications are capable to work with the JSON details needed to send and receive data. Having multiple client apps store the needed configuration is an overhead that can be avoided by using a central service (azure function) for it. Using "OpenAPI" (don't confuse it with OpenAI) the potential group of consumers is extended even into low-code space, for example utilizing custom connectors to give access to Logic Apps or other Cloud Workflows. Furthermore when updating how to access the OpenAI services that can be done without updating all client applications, even by using CI/CD pipelines. Last but not least, Azure Functions can be used from private Endpoints. While a client application may not have access to the internet, it can have access to the function, which in turn can make a limited call to the internet.

Security

Access and billing on OpenAIs end are done through APIKeys. You can make multiple of those, but then it gets tricky. Which key is in whose hands? When someone leaves the project, which key do I need to rotate? Did someone leak the key into a source code system by mistake? How will the client applications, which might be developed outside the organization, store and secure the APIKey?

By using a central entity like the Azure Function all client applications can be secured with Azure AD, using Application Registrations with certificates, Managed Identity or any other security mechanism.

Cost

Azure Functions are notoriously cheap. By outsourcing the access to OpenAI you save up the working hours needed to implement and update the access to OpenAI with every service and workflow, instead offering a streamlined one-stop solution.

By using your own security concept, you can also impose usage limits per consumer, for example, an application registration can only use a specific amount of tokens per 30 days rolling, therefore eliminating the danger of overuse.

The Code

The Function itself is straightforward: we accept the message to summarize as the HTTP POST Body and return the summary as text. Using Swagger (OpenAPI) we expose the endpoint more easily as well.

 [FunctionName("Summarize")]
        [OpenApiOperation(operationId: "Summarize", tags: new[] { "Summarize" })]
        [OpenApiRequestBody(contentType: "text/plain", bodyType: typeof(string), Required = true, Description = "The text to be summarized")]
        [OpenApiResponseWithBody(statusCode: HttpStatusCode.OK, contentType: "text/plain", bodyType: typeof(string), Description = "The summary")]
        public async Task<IActionResult> Run(
            [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequest req)
        {
            _logger.LogInformation("C# HTTP trigger function processed a request.");

            string requestBody = await new StreamReader(req.Body).ReadToEndAsync();

            try
            {
                var summary = await CreateSummaryAsync(requestBody);
                return new OkObjectResult(summary);
            }
            catch(Exception e)
            {
                _logger.LogError(e, e.Message);
                return new BadRequestObjectResult(e.Message);
            }        
        }

CreateSummary is where the access to OpenAI is happening. This code is a function that sends a request to the OpenAI API to generate a summary of the input text. The function takes a string as input and returns a string as output.

The function first concatenates the prompt "Summarize this text in one sentence: " with the input text. It then retrieves the OpenAI API key from the environment variables and creates an HTTP client object with the authorization header set to the API key.

The function then creates an anonymous object with the input parameters for the API request and serializes it to a JSON string. It then sends an HTTP POST request to the OpenAI API endpoint with the request content set to the serialized JSON string. If the request is successful, it reads the response content and parses it as a JSON object. It extracts the summary text from the response and returns it as a string. If an exception is thrown during any of these operations, the function catches it and throws a new exception with an informative message.

private async Task<string> CreateSummaryAsync(string text)
        {
            text = "Summarize this text in one sentence: " + text;
            string apiKey = Environment.GetEnvironmentVariable("OpenAiApiKey");
            if (string.IsNullOrEmpty(apiKey)) throw new Exception("Please add 'OpenAiApiKey' to your environment variables");            

            HttpClient http = new HttpClient();
            http.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", apiKey);

            var data = new
            {
                prompt = text,
                model = "text-davinci-003",
                max_tokens = 100,
                temperature = 0.9,
                top_p = 1,
                frequency_penalty = 0,
                presence_penalty = 0.6,
                stop = ""
            };

            StringContent content = new StringContent(JsonSerializer.Serialize(data), Encoding.UTF8, "application/json");

            try
            {
                var request = await http.PostAsync("https://api.openai.com/v1/completions", content);
            request.EnsureSuccessStatusCode();            
            string resultContent = await request.Content.ReadAsStringAsync();

                var root = JsonNode.Parse(resultContent).AsObject();
                var choices = root["choices"].AsArray();
                return choices[0]["text"].ToString();
            }catch(Exception e)
            {
                throw new Exception("Error parsing response from OpenAI", e);
            }
        }

The result

The default swagger UI generated from our function looks like this:

And after execution, the summary is returned to the client

The coresponding CURL command would look like this, for reference

curl -X POST "http://localhost:7251/api/Summarize" -H  "accept: text/plain" -H  "Content-Type: text/plain" -d "Dear Olivia,The solar energy conference went great. New Horizon Manufacturing wants to meet with us to talk about our photovoltaic window system we’re about to launch.I think it would be great to talk this Tuesday.Best,Allison"

Conclusion

This post shows how to do a wrapped call to OpenAI APIs. If you want to know more about what to do as a developer with chatGPT, this post might interest you: https://blog.jenscaasen.cloud/12-ways-to-make-chatgpt-your-personal-junior-developer-with-examples