I’ve been experimenting with different large language models and have been using Verida’s Personal Agent Kit to connect a command line chatbot to my emails, Telegram data, and other private information. During my experiments, I noticed something unexpected in how the various models process tools.
I started by running my command line app configured to use Claude 3.5 from Anthropic. I asked it to list the tools it can access and then inquired about the parameters for the query email tool. Claude correctly returned a detailed list of parameters—such as selector, field, sort, limit, skip, and count—demonstrating its full support for filtering and sorting.
Next, I switched my configuration to use OpenAI’s GPT-4. After updating the settings, I ran the same command to view the available tools. While GPT-4 did list the tools (with names or descriptions), when I asked for the parameters of the query email tool, it only returned four: field, limit, skip, and count. Notably, it omitted the selector and sort parameters. Without these, it's difficult to filter emails (for example, by using a regular expression or distinguishing between sent and received emails) effectively.
I also tested Llama 3.3 70B. When configured with Llama, the model initially tried to execute the tools rather than just listing them. After a second attempt—making sure it only listed the parameters—it successfully returned the complete set, including selector and sort.
This inconsistency, particularly with GPT-4 and other OpenAI models, limits their utility in scenarios where full tool parameter support is required. I’m curious to know if others have experienced similar issues or have found workarounds.
