We dive into GPT3.5 and GPT4

We offer access to GPT 3.5, GPT 4, and GPT 4 turbo in Smitty.ai. We dive deeper into the context limits and costs. And on how Smitty handles these things. We start explaining how the system is set up by Open Ai, at the end of the post we dive deeper in the Smitty part of this.

GPT 3.5

Release: March 15, 2022

Knowledge of events: Up to September 2021

Parameters: 175 billion

Input: Text only

Context limit: 16000 tokens

Factual responses: Occasional errors

GPT 4

Release: March 14, 2023

Knowledge of events: Up to April 2023

Parameters: 100 trillion

Input: Text and images

Context limit: 128000 tokens

Factual responses: 40% more accurate

Tokens and words broken down

The AI models we currently support operate with tokens, which roughly translate to about 4 characters of text for regular English text. This roughly translates to 3/4 of a word. So, 100 tokens is approximately 75 words.

We did some research to know the maximum context limits it can handle and on how much words this is approximately.

Number of words and tokens:

GPT 3.5: 4,096 tokens, approximately 3,000 words

GPT 3.5: 16,000 tokens, approximately 11,680 words

GPT 4: 32,000 tokens, approximately 23,260 words

GPT 4 turbo: 128,000 tokens, approximately 93,440 words

Additionally, the training data for GPT-3.5 Turbo, unlike GPT-4, is up-to-date from April 2023, while the free version of GPT 3.5 relies only on data up to September 2021. Essentially, this new version allows users to input and consider longer texts when composing prompts or questions they seek answers to. Consequently, ChatGPT can now account for more intricate details and data, resulting in more accurate and comprehensive responses.

Costs

Source information as of February 2023              

input costs              output costs

GPT 3.5 turbo 4k              0.0015 cents per 1000 tokens        0.0020 cents/ 1k tokens

GPT 3.5 turbo 16k            0.0005 cents per 1000 tokens        0.0015 cents/ 1k tokens

GPT 4 32k                             0.06 cents per 1000 tokens             0.12 cents/ 1k tokens

GPT 4 turbo 128k              0.01 cents per 1000 tokens          0.03 cents/ 1k tokens

Costs GPT 4 turbo

So, a quick calculation shows that a large document of 128,000 tokens with GPT 4 turbo costs $1.28 to process, and the output is the AI's response, which can also be lengthy. The costs can vary from $1 to $2 depending on the length of the response. With just one question, this can quickly amount to $3. Let alone a conversation where previous answers and context are taken into account. That quickly adds up.

Costs GPT 3.5 turbo 16k

The most cost-effective GPT is the 3.5 turbo with a 16k context. A calculation shows that for 16,000 tokens / 1000 tokens * 0.0005 cents = 0.008 cents. Chatting with this bot is very cheap.

For the latest prices, check the OpenAI website: https://openai.com/pricing

Word count limits of AI and how Smitty handles them

As you can see, there is a limit on the number of words an AI can process. We have a solution for this within Smitty and split documents into parts. Within Smitty, it is possible to split a document into large chunks of up to 10,000 characters. Due to how Smitty is set up, entire documents are not sent to the AI, but rather the relevant part is retrieved when the question is asked.

Because we work with a smart search system for relevant information, this saves costs. This saves costs in using our credits for the OpenAI integration but also saves costs from the business because there is no need to program a Q&A.

You just upload your document / data and Smitty is trained with this information.