The rapid rise of artificial intelligence is causing more and more people to ask: “Will AI replace my job?” Among them are programmers who see the enormous potential of AI in software development. But what can be done to avoid falling out of favor and being replaced by AI?
It has been suggested in the business community that if AI replaces humans, it will only replace those who do not learn to harness it. What does this mean for us as developers? First, we must stay abreast of technological innovations to operate more efficiently and deliver innovative solutions.
If you’re wondering how the developer’s at DLabs.AI use AI—and by extension, how YOU can use AI—you’re in the right place. By reviewing how we created an internal project called SugarAssist, a diabetic assistant, you’ll learn where and how we use AI in our daily work.
Let’s dive in!
Understanding the Basics: GPT vs. ChatGPT
Before diving too deep, it’s crucial to grasp the foundational concepts of artificial intelligence, language models, and the specific distinctions between GPT and ChatGPT. Doing so will provide a clearer perspective on the content and enhance your overall understanding.
What is GPT?
In a nutshell, GPT—which stands for Generative Pre-trained Transformer—is a type of artificial intelligence language model developed by OpenAI. It’s designed to generate coherent and contextually relevant text based on a prompt. Its iterations, such as GPT-3, GPT-3.5, GPT-4, have been widely recognized for their ability to produce human-like text.
What is ChatGPT?
ChatGPT, on the other hand, is a specialized app built using the GPT model for conversational interactions. It’s fine-tuned specifically for generating conversational responses, making it ideal for tasks like creating chatbots or virtual assistants.
To get a more detailed explanation and a clearer definition of these terms (and other AI-related words), I recommend referring to the definitions in our glossary “AI for beginners“.
Two Primary Ways to Use AI in a Software Developer’s Work
To set the stage, I’ll say that AI in a developer’s work comes in two forms:
- Language Model-Powered Tools for Enhanced Productivity: Developers use tools based on language models to optimize processes and thus increase their productivity. The most popular tools among our developers are Copilot and ChatGPT.
- Creating Apps using AI: Developers use AI in building innovative applications. In our case, during the development of our GPT-based diabetes assistants.
Case study: Creating an LLM-based application using LLM-based tools
To best illustrate how LLM-based tools can be used, let me use a real-life example. If this is your first time on our website, you may not know that our company’s overarching goal is to create a comprehensive diabetes management tool.
Our flagship product, Suguard, is still under development, and one of the challenges we faced was to better understand the needs and problems of people with type one diabetes.
Step 1: Using ChatGPT to create a survey
We began our work with weekly meetings with the entire development team. During these meetings, we brainstormed ideas and set the direction of the work for the following week. One goal was to get information from people with type I diabetes, so we decided to create a survey.
ChatGPT was helpful in this task, generating a list of questions that we used to create the survey. We then sent it to people with diabetes on various community forums and to people in our database who were willing to participate in product development. These people were either type I diabetics or caregivers of children with recently diagnosed type I diabetes.
Step 2: Looking for solutions to automate data collection
Despite a significant number of responses, we felt the survey failed to provide us with enough data. That’s why we decided to create a tool to collect data on the needs of people with type I diabetes. This became our “Diabetic Assistant,” an agent based on a language model, built to solve two problems:
- Lack of sufficient data to create a comprehensive diabetes management tool: the assistant was supposed to help collect this data.
- Lack of knowledge and awareness about diabetes among children with recently diagnosed type I diabetes and their parents: the assistant was to help educate and increase confidence in making daily treatment decisions.
Step 3: Preparing Product Requirements
This was followed by event storming, a workshop in which one creates a model of events using symbols and cards to better understand what events and processes affect the functioning of such a tool.
Once we had a robust product requirement document, including all the project’s guidelines and assumptions (and so we knew what we were creating and why), we moved on to picking tools to support the development of the assistant itself.
Step 4: Tool selection
4.1. Low-code platform selection
Low-code platforms seemed an attractive option to speed up and simplify the application development process. We settled on Flutter Flow, a low-code solution to build a fully functioning app (and you should note that we were able to build most of the functionality without any advanced knowledge of this tool or the Dart language, which this tool uses).
That said, when we needed to write a custom action in Dart, we decided to use ChatGPT to generate it.
The action would make requests to the API based on the OpenAPI documentation, and subsequently execute the appropriate action based on the input provided.
You can see our guide ‘Story of creating action for flutter flow’ to learn how to specify the command for ChatGPT to get the expected result. The instructions there came after a lot of experimentation, with us eventually landing on the expected result.
In our case, with a personalized application, the low code option proved too limited. Instead, we decided to use this solution for blogs or e-commerce applications.
4.2. Large Language Model selection
We knew we wanted to use a language model. We tried Vectara, an LLM-based platform for search. Vectara offers a complete search process, from extraction and indexing to retrieval, re-sorting and calibration. However, it turned out to lose too much compared to the GPT model that was tested next.
The assistant needed to be able to answer diabetes-related questions, as well as provide personalized advice. So we decided to use off-the-shelf language models such as GPT 3, GPT 3.5 and GPT 4, which we integrated into our application. They differed significantly in understanding prompts and performance.
More precisely: the higher version did better with understanding prompts, and the lower version with performance. But why settle on GPT? Unlike other models that are mainly discriminative (e.g., BERT), GPT is a generative model.
This means it’s specifically designed to create new text, which is crucial for a chat assistant that needs to generate responses in real-time.
Step 5: LLM customization
5.1. Defining Input prompts
With the model selected, we can focus on adapting it to our needs. First of all, we need to know what data our model should use. In our case, we wanted the knowledge provided to users to be proven, so we decided to use our eBook, Sugarscout, a guide for newly diagnosed children and their caregivers. We transferred the key content to the language model by defining it in prompts.
Input prompts are input queries, questions or instructions that allow communication with the model, giving it specific questions or guidelines on what to generate. Input prompts and guidelines are interactions with an already trained model to elicit answers or generate text.
This isn’t an easy task, as only one of the subsequent attempts has been successful. You can learn how precise the prompts should be from our guide on GitHub: Story of ChatGPT prompt accuracy.
5.2. Defining how the model should generate responses
In addition to introducing prompts, it’s important to define rules, guidelines, or constraints to determine how the model should generate responses. Rules control the style, tone, length, and other characteristics of the content generated.
They can be ethical rules, writing style guidelines, content restrictions or other conditions. For example, a rule may prohibit the model from generating offensive or non-compliant content. In our case, defining rules became very important, because our assistant was prividing medical advice in reponse to queries such as the amount of insulin to administer.
In such a case, it was important to highlight that the assistant could only help suggest a dosage by showing how it should be calculated. But as it’s not a medical professional, it says that in the event of any doubt, the user should consult a doctor.
5.2. Retraining the model
It turned out that defining prompts and rules was not enough. After encountering repeated incorrect responses and finding ways to circumvent the rules, we decided to retrain our diabetic assistant.
Here are several examples of how to retrain a model:
- Transferable Learning: a technique that involves using the knowledge and skills of trained models on one task to improve the model’s performance on another task. In transfer learning, certain layers of the model are adjusted to better suit the new task, similar to fine-tuning. This is especially useful if you have access to a pre-trained model that’s close to your new task.
- Regularization: a technique that involves adding a penalty for overly complex models or models that show signs of over-fitting the training data.
- Fine-tuning: focuses on training a model on a specific data set to better adapt it to a specific task. This relies on the fact that when you have a pre-trained language model and want it to be a medical specialist, you can train it on medical texts. After this process, the model will be better able to handle questions and topics related to medicine.
- Embedding Vectors: these are numerical representations that allow abstract, or categorical data, such as words, tokens, images or other elements, to be transformed into numbers that can be processed by machine learning models. These representations are created in the learning process, and allow certain relationships and meanings between data to be preserved.
Embeddings allow models to effectively learn and understand these elements. With embeddings, you can adapt these representations to your specific task by training them together with the model. That is, for tasks where understanding the semantics of text or data is important, embeddings can help you get a better representation.
They can also help the model produce more varied and meaningful answers compared to when the model would be based on rules and prompts alone.
Why did we choose data embeddings?
After delving into the various ways to train the model, we decided to use embeddings, and we’re currently working on this.
In our case, this way of post-training seems the most suitable owing to the following reasons:
- Task specifics: Our task is to provide information and support to children and parents of children with type I diabetes. This requires understanding and empathy towards this particular health problem. Embeddings can help create better data representations, which is useful for semantic analysis and content comprehension tasks, which is important in educational and support tasks.
- The multifaceted nature of the topic: Type I diabetes is a disease that has many facets, such as diet, glucose monitoring, treatment, psychological aspects, and more. Embeddings allow for the creation of representations for different aspects of the topic, which can help provide personalized and relevant answers to different user questions and needs.
- Education and empathy: Embeddings can help understand users’ content and questions on a semantic level, which is key to education and support for a disease such as diabetes. Additionally, the use of embeddings can help create more empathetic responses that are tailored to users’ emotional needs.
- Complex content: Diabetes is a complex topic, and the knowledge related to it is wide and varied. Embedding can help create content data representations that allow for more comprehensive and understandable answers to user questions.
- Customization for specific questions: Embeddings allow the model to be tailored to specific terminology and questions related to diabetes, which can lead to more precise answers.
Interestingly, we also chose this way when creating the Student Agent, whose creation process is described in the article “10 Steps to Successfully Implementing GPT in Your Project”.
It’s important to consider ethics and patient privacy when dealing with medical applications. The model and responses should be controlled to ensure the accuracy of the information and avoid inappropriate content. As this approach may not always prove suitable, it’s important to stay aware that the model may benefit from using other approaches in the future.
Step 6: Testing applications with real users
Next, we moved on to a key part of the chat application development process. Why? Three reasons:
- To check the quality of the interaction (i.e., whether our assistant adequately understands users’ questions and whether it provides reliable, well formulated answers);
- To detect errors and gaps in the prompts introduced;
- To evaluate the performance of the assistant in real conditions.
Testing in our case was done through organized bug-bash sessions, where users tried to catch as many irregularities as possible; from bugs in the UI to bugs in application responses and performance. During testing, it was helpful to use the GPT chat, which could generate questions of a different nature, through which we could see if there were errors in the prompts and rules introduced.
During these sessions, there can be amusing situations were an assistant who supposedly can’t tell jokes because of a guideline ends up telling jokes because of the way a question was worded — which, by the way, was why we decided to give the model additional training.
Step 7: Fixing bugs, and release the application to the first group of users
After several such bug-bashing sessions—followed by fixes and optimizations—we sent a first version to a group of target users. In our case, these were people with type I diabetes who left contact information when surveyed in the initial phase of the project.
Now, while bug-bush sessions are designed to catch bugs (with bugs reported and tasks then created to fix the bug), when an application is used by a real user, we find bugs in two ways: when the user reports an error (which doesn’t happen very often), or when we look through the database and see what questions were asked and what answers were given.
We had one instance where a user asked if he would be able to go back to his questions and the answers received, to which SugarAssist said, “Yes” — and this just isn’t true. As a result, we saw the need to implement this functionality or provide clearer instructions for the chat.
So as you can see: testing leads to optimization, with each session helping you create an assistant that better meets the needs of the user.
Step 8: Publishing the app online
The final stage is getting the app live, and SugarAssist is now available online, so I encourage you to take go check it out. We’re still testing and improving it so that it can better serve diabetic children and their caregivers, alongside providing data that helps us better support diabetics in the future.
Want to learn more about GPT and other Large Language Models?
If you’d like to learn more about the process of creating assistants using the GPT model, I encourage you to read “10 Steps to Successfully Implementing GPT in Your Project” where we run you through a case study about building a “student assistant.”
And if you want to know more about how using tools based on language models can help in your work (and what you can do to be more productive), I prepared an article called “Tools based on language models in a developer’s daily work.”
Feel free to give them both a read and let us know what you think.
AI can be a developer’s best friend
Artificial Intelligence, including language models, can help in many aspects of a developer’s job, automating tasks, speeding up processes, and helping you create more advanced solutions.
Still, developers will always have a key role in IT projects, whether creating algorithms, designing system architecture, or solving complex problems. That’s why we love talking about the synergy between AI and developers, rather than focusing on one replacing the other.
Now, if you need help with an LLM implementation, take a look at our GPT integration offer and schedule a free consultation with our AI experts today.
