An Introduction to OpenAI Perform Calling | by David Hundley | Jul, 2023

No extra unstructured knowledge outputs; flip ChatGPT’s completions into structured JSON!

Just a few months in the past, OpenAI launched their API to most people, which excited many builders who needed to utilize ChatGPT’s outputs in a scientific method. As thrilling has this has been, it’s equally been a little bit of a nightmare since we programmers are inclined to work within the realm of structured knowledge varieties. We like integers, booleans, and lists. The unstructured string might be unwieldy to cope with, and with a view to get constant outcomes, a programmer is required to face their worst nightmare: growing an everyday expression (Regex) for correct parsing.

After all, immediate engineering can really assist fairly a bit right here, nevertheless it’s nonetheless not good. For instance, if you wish to have ChatGPT analyze the sentiment of a film evaluate for positivity or negativity, you would possibly construction a immediate that appears like this:

immediate = f'''
Please carry out a sentiment evaluation on the next film evaluate:
{MOVIE_REVIEW_TEXT}
Please output your response as a single phrase: both "Optimistic" or "Damaging".
'''

This immediate actually does fairly decently, however the outcomes aren’t exactly constant. For instance, I’ve seen ChatGPT produce outputs that seem like the next by way of the film sentiment instance:

• Optimistic

• constructive

• Optimistic.

This may not appear to be an enormous deal, however on this planet of programming, these are NOT equal. Once more, you may get round a less complicated instance like this with a little bit of Regex, however past the truth that most individuals (together with myself) are horrible at writing common expressions, there are merely some cases the place even Regex can’t parse the data appropriately.

As you’ll be able to inform, programmers have been hoping that OpenAI would add performance to help structured JSON outputs, and OpenAI has delivered within the type of perform calling. Function calling is precisely because it sounds: it permits ChatGPT to provide arguments that may work together with a customized perform in a way that makes use of structured knowledge varieties. Yup, no extra fancy immediate engineering and Regex to cross your fingers and hope you get the correct end result. On this publish, we’ll cowl the best way to make use of this new performance, however first, let’s begin with an instance of how we used to try to provide structured knowledge outputs with immediate engineering and Regex.

Earlier than we soar into the majority of our publish, please permit me to share a link to this Jupyter notebook in my GitHub. This pocket book incorporates all of the code I will probably be working (and extra) as a part of this weblog publish. Moreover, I might encourage you to check out OpenAI’s official function calling documentation for something that I could not cowl right here.

To reveal what we used to do within the “pre-function calling days”, I wrote a small little bit of textual content about myself, and we’ll be utilizing the OpenAPI to extract bits of data from this textual content. Right here is the “About Me” textual content we’ll be working with:

Let’s say I need to extract the next bits of data from that textual content:

• Identify

• Job title

• Firm

• Variety of youngsters as an integer (That is necessary!)

• Automotive make

• Automotive mannequin

• Favourite online game collection

Right here’s how I might engineer a few-shot immediate with a view to produce a structured JSON output:

# Engineering a immediate to extract as a lot data from "About Me" as a JSON object
about_me_prompt = f'''
Please extract data as a JSON object. Please search for the next items of data.
Identify
Job title
Firm
Variety of youngsters as a single integer
Automotive make
Automotive mannequin
Favourite online game collection
That is the physique of textual content to extract the data from:
{about_me}
'''
# Getting the response again from ChatGPT (gpt-3.5-turbo)
openai_response = openai.ChatCompletion.create(
mannequin = 'gpt-3.5-turbo',
messages = [{'role': 'user', 'content': about_me_prompt}]
)
# Loading the response as a JSON object
json_response = json.masses(openai_response['choices'][0]['message']['content'])
json_response

Let’s take a look at how ChatGPT returned this completion to me:

As you’ll be able to see, this really isn’t unhealthy. But it surely’s not best and will show to be dangerous for the next causes:

• We aren’t assured that OpenAI’s response will present a clear JSON output. It might have produced one thing like “Right here is your JSON:” adopted by the JSON output, that means that with a view to use json.masses() to parse the string right into a JSON object, we’d first need to strip out that little little bit of textual content that opens the response.

• We aren’t assured that the keys within the key-value pairs of the JSON object will probably be constant from API name to API name. Recall the instance from above of the three cases of the phrase Optimistic. That is exactly the identical danger you run attempting to have ChatGPT parse out keys via few-shot immediate engineering. The one method you would possibly lock this down is with Regex, which comes with its personal baggage as we already mentioned.

• We aren’t assured to obtain our responses within the correct knowledge sort format. Whereas our immediate engineering to extract variety of youngsters did parse into a correct integer, we’re on the mercy of crossing our fingers and hoping we get that constant end result for each API name.

We might summarize these points right into a single assertion: With out perform calling, we aren’t assured to get constant outcomes which can be necessary for the precision required for systematic implementation. It’s a nontrivial challenge that may be very difficult to treatment via immediate engineering and common expressions.

Now that we’ve constructed an instinct round why getting structured outputs from ChatGPT was previously problematic, let’s transfer into trying on the new perform calling functionality launched by OpenAI.

Perform calling is definitely a little bit of a misnomer. OpenAI just isn’t really working your code in a real perform name. Reasonably, it’s merely organising the structured arguments you’d have to execute your personal customized features, and I’d argue that is most well-liked conduct. Whilst you could be considering that it doesn’t make sense that the OpenAI API isn’t executing your customized perform, take into account that with a view to try this, you’d need to move that perform code into ChatGPT. This perform code most likely incorporates proprietary data that you’d NOT need to expose to anyone, therefore why it’s good that you just don’t really need to move this code to utilize OpenAI’s perform calling.

Let’s soar into an instance of the best way to allow perform calling with a single customized perform. Utilizing our “About Me” pattern textual content from the earlier part, let’s create a customized perform referred to as extract_person_info. This perform wants simply three bits of data: individual title, job title, and variety of youngsters. (We’ll revisit extracting the remainder of the data within the subsequent part; I simply need to begin easier for now.) This practice perform is deliberately quite simple and can merely take our arguments and print them collectively in a single string. Right here’s the code for this:

def extract_person_info(title, job_title, num_children):
'''
Prints fundamental "About Me" data
Inputs:
- title (str): Identify of the individual
- job_title (str): Job title of the individual
- num_chilren (int): The variety of youngsters the guardian has.
'''
print(f'This individual's title is {title}. Their job title is {job_title}, they usually have {num_children} youngsters.')

With a view to make use of perform calling, we have to arrange a JSON object in a particular method that notes the title of our customized perform and what knowledge parts we hope ChatGPT will extract from the physique of the textual content. Due to the specificity on how this JSON object ought to look, I might encourage you reference OpenAI’s developer documentation if you wish to know any particulars that I don’t cowl right here.

(Observe: Within the OpenAI documentation, I observed one aspect within the JSON object referred to as required that seemingly signifies {that a} parameter have to be current for ChatGPT to correctly acknowledge the perform. I attempted testing this out, and both this isn’t how this performance works or I did one thing flawed. Both method, I transparently don’t know what this required parameter signifies. )

Right here is how we have to construction our JSON object to utilize our customized perform:

my_custom_functions = [
{
'name': 'extract_person_info',
'description': 'Get "About Me" information from the body of the input text',
'parameters': {
'type': 'object',
'properties': {
'name': {
'type': 'string',
'description': 'Name of the person'
},
'job_title': {
'type': 'string',
'description': 'Job title of the person'
},
'num_children': {
'type': 'integer',
'description': 'Number of children the person is a parent to'
}
}
}
}
]

You’re most likely already aware of JSON syntax, though let me draw consideration for a second to the info sort related to every property. In case you are a Python developer like myself, remember that the info typing for this JSON construction is NOT instantly equal to how we outline knowledge buildings in Python. Typically talking, we are able to discover equivalencies that work out alright, however if you wish to know extra concerning the particular knowledge varieties related to this JSON construction, check out this documentation.

Now we’re able to make our API name to get the outcomes! Utilizing the Python shopper, you’ll discover the syntax is similar to how we receive completions on the whole. We’re simply going so as to add some further arguments into this name that signify our perform calling:

# Getting the response again from ChatGPT (gpt-3.5-turbo)
openai_response = openai.ChatCompletion.create(
mannequin = 'gpt-3.5-turbo',
messages = [{'role': 'user', 'content': about_me}],
features = my_custom_functions,
function_call = 'auto'
)
print(openai_response)

As you’ll be able to see, we merely move in our checklist of customized features (or in our case for now, our singular customized perform) because the features parameter, and also you’ll additionally discover an extra parameter referred to as function_call that we’ve set to auto. Don’t fear about this for now as we’ll revisit what this auto piece is doing within the subsequent part.

Let’s run this code and try the total API response from ChatGPT

For essentially the most half, this response seems to be the identical as a non-function name response, however now there’s an extra discipline within the response referred to as function_call, and nested beneath this dictionary are two further gadgets: title and arguments. title signifies the title of our customized perform that we’ll be calling with ChatGPT’s output, and arguments incorporates a string that we are able to load utilizing json.masses() to load our customized perform arguments as a JSON object.

Discover now that we’re getting rather more consistency than we had been in our pre-function calling methodology. Now we might be assured that the keys of the key-value pairs WILL be constant, and the info varieties WILL be constant. No want for fancy immediate engineering or common expressions!

That’s the core of OpenAI’s perform calling! After all, this was a really simplistic instance to get you going, however you most likely have further questions. Let’s cowl these on this subsequent part.

The earlier part lined a quite simple instance of the best way to allow perform calling, however should you’re like me, you most likely have some further questions past this level. Naturally, I can’t cowl all these questions, however I do need to cowl two large ones which can be barely extra superior than what we lined within the earlier part.

What if the immediate I submit doesn’t include the data I need to extract per my customized perform?

In our authentic instance, our customized perform sought to extract three very particular bits of data, and we demonstrated that this labored efficiently by passing in my customized “About Me” textual content as a immediate. However you could be questioning, what occurs should you move in some other immediate that doesn’t include that data?

Recall that we set a parameter in our API shopper name referred to as function_call that we set to auto. We’ll discover this even deeper within the subsequent subsection, however what this parameter is basically doing is telling ChatGPT to make use of its finest judgment in determining when to construction the output for one among our customized features.

So what occurs after we submit a immediate that doesn’t match any of our customized features? Merely put, it defaults to typical conduct as if perform calling doesn’t exist. Let’s check this out with an arbitrary immediate: “How tall is the Eiffel Tower?”

As you’ll be able to see, we’re getting a typical “Completions” output although we handed in our customized perform. Naturally, this is smart since this arbitrary Eiffel Towel immediate incorporates not one of the particular data we’re searching for.

What if I need to move a number of customized features and a few of them have overlapping parameters?

In brief, ChatGPT intelligently handles this with out a drawback. The place we beforehand handed in a single customized perform as primarily a listing of Python dictionaries, we simply have to hold including further Python dictionaries to this similar checklist, every representing its personal distinct perform. Let’s add two new features: one referred to as extract_vehicle_info and one other referred to as extract_all_info. Right here’s what our adjusted syntax seems to be like:

# Defining a perform to extract solely car data
def extract_vehicle_info(vehicle_make, vehicle_model):
'''
Prints fundamental car data
Inputs:
- vehicle_make (str): Make of the car
- vehicle_model (str): Mannequin of the car
'''
print(f'Automobile make: {vehicle_make}nVehicle mannequin: {vehicle_model}')
# Defining a perform to extract all data offered within the authentic "About Me" immediate
def extract_vehicle_info(title, job_title, num_children, vehicle_make, vehicle_model, company_name, favorite_vg_series):
'''
Prints the total "About Me" data
Inputs:
- title (str): Identify of the individual
- job_title (str): Job title of the individual
- num_chilren (int): The variety of youngsters the guardian has
- vehicle_make (str): Make of the car
- vehicle_model (str): Mannequin of the car
- company_name (str): Identify of the corporate the individual works for
- favorite_vg_series (str): Individual's favourite online game collection.
'''
print(f'''
This individual's title is {title}. Their job title is {job_title}, they usually have {num_children} youngsters.
They drive a {vehicle_make} {vehicle_model}.
They work for {company_name}.
Their favourite online game collection is {favorite_vg_series}.
''')
# Defining how we wish ChatGPT to name our customized features
my_custom_functions = [
{
'name': 'extract_person_info',
'description': 'Get "About Me" information from the body of the input text',
'parameters': {
'type': 'object',
'properties': {
'name': {
'type': 'string',
'description': 'Name of the person'
},
'job_title': {
'type': 'string',
'description': 'Job title of the person'
},
'num_children': {
'type': 'integer',
'description': 'Number of children the person is a parent to'
}
}
}
},
{
'name': 'extract_car_info',
'description': 'Extract the make and model of the person's car',
'parameters': {
'type': 'object',
'properties': {
'vehicle_make': {
'type': 'string',
'description': 'Make of the person's vehicle'
},
'vehicle_model': {
'type': 'string',
'description': 'Model of the person's vehicle'
}
}
}
},
{
'name': 'extract_all_info',
'description': 'Extract all information about a person including their vehicle make and model',
'parameters': {
'type': 'object',
'properties': {
'name': {
'type': 'string',
'description': 'Name of the person'
},
'job_title': {
'type': 'string',
'description': 'Job title of the person'
},
'num_children': {
'type': 'integer',
'description': 'Number of children the person is a parent to'
},
'vehicle_make': {
'type': 'string',
'description': 'Make of the person's vehicle'
},
'vehicle_model': {
'type': 'string',
'description': 'Model of the person's vehicle'
},
'company_name': {
'type': 'string',
'description': 'Name of the company the person works for'
},
'favorite_vg_series': {
'type': 'string',
'description': 'Name of the person's favorite video game series'
}
}
}
}
]

Discover particularly how the extract_all_info covers a few of the similar parameters as our authentic extract_person_info perform, so how does ChatGPT know which one to pick out? Merely put, ChatGPT seems to be for the very best match. If we move in a immediate that incorporates all of the arguments wanted for the extract_all_info perform, that’s the one it’ll choose. But when we simply move in a immediate that incorporates both simply easy details about me or a immediate about my car, it’ll leverage the respective features that try this. Let’s execute that in code right here with just a few samples:

• Pattern 1: The unique “About Me” textual content. (See above.)

• Pattern 2: “My title is David Hundley. I’m a principal machine studying engineer, and I’ve two daughters.”

• Pattern 3: “She drives a Kia Sportage.”

With every of the respective prompts, ChatGPT chosen the right customized perform, and we are able to particularly word that within the title worth beneath function_call within the API’s response object. Along with this being a useful strategy to establish which perform to make use of the arguments for, we are able to programmatically map our precise customized Python perform to this worth to run the right code appropriately. If that doesn’t make sense, maybe this in code would make this extra clear:

# Iterating over the three samples
for i, pattern in enumerate(samples):
print(f'Pattern #{i + 1}'s outcomes:')
# Getting the response again from ChatGPT (gpt-3.5-turbo)
openai_response = openai.ChatCompletion.create(
mannequin = 'gpt-3.5-turbo',
messages = [{'role': 'user', 'content': sample}],
features = my_custom_functions,
function_call = 'auto'
)['choices'][0]['message']
# Checking to see {that a} perform name was invoked
if openai_response.get('function_call'):
# Checking to see which particular perform name was invoked
function_called = openai_response['function_call']['name']
# Extracting the arguments of the perform name
function_args = json.masses(openai_response['function_call']['arguments'])
# Invoking the correct features
if function_called == 'extract_person_info':
extract_person_info(*checklist(function_args.values()))
elif function_called == 'extract_vehicle_info':
extract_vehicle_info(*checklist(function_args.values()))
elif function_called == 'extract_all_info':
extract_all_info(*checklist(function_args.values()))

**Beware one factor**: Within the spirit of full transparency, I needed to run that code there a number of instances to get it to provide like that. The difficulty is that as a result of the extract_person_info and extract_all_info are extra comparable in nature, ChatGPT stored complicated these for each other. I suppose the lesson to be realized right here is that your features must be extracting distinct data. I additionally solely examined utilizing gpt-3.5-turbo, so it’s attainable {that a} extra highly effective mannequin like GPT-4 might have dealt with that higher.

I hope you’ll be able to see now why perform calling might be so highly effective! In relation to constructing purposes that leverage Generative AI, this sort of perform calling is a godsend for programmers. By not having to fret a lot now concerning the output JSON construction, we are able to now focus our time on constructing out different elements of the applying. It’s an superior time to be working on this house

The post An Introduction to OpenAI Perform Calling | by David Hundley | Jul, 2023 appeared first on AIPressRoom.