Methods for building a good AI product

5 months ago I started developing a product around content creation assistance for LinkedIn for Lumeo (lumeo.me)

I was able to learn a lot of lessons from it in terms of using AI in a product and today I am sharing them with you

First of all, when I started, I knew how to make fairly decent prompts, but nothing incredible.

My goal was to offer a quality LinkedIn post with a human style and a style that corresponds to the user in question

Here are the steps I went through:

First of all, I started with prompt engineering based on Chat GPT-4, so I took a look at the prompting guide which allowed me to open up some ideas on how I was going to be able to design a good prompt.

What I got from it:

• the few shots example really helps to calibrate the writing style
• giving the role and status to the assistant allows to put it in a good context
• giving as much detail as possible on what you want in output allows to avoid hallucinations and to have more coherent results

After that it was necessary to analyze that the results we had were better than the old ones for that we set up thanks to promptfoo: a tool to compare against a prompt and/or a model the answers that return.

Our case was a bit complicated for the use of promptfoo because we are looking for something authentic so nothing should be similar between two prompts but we managed to set conditions to test and be sure that we were going in the right direction

After that I looked at the fine tuning part which could have been a very good solution for my case but in the end it was not really conclusive, it must be used for very specific cases and in my case it required too much work to have something satisfactory. So I discarded this track to concentrate on the rest.

In the meantime, I developed a Chrome extension that allows me to retrieve the user's LinkedIn publications to imitate their writing style as best as possible (a bit of reverse engineering and that's it)

Then, to continue improving the publication generation that I already liked, I worked on setting up a RAG (retrieval augmented generation) that allows you to provide a knowledge base (in a vector database) where you can deposit all types of files, these are cut into chunks and transformed into vectors. I inserted a lot of documents on the basics of copywriting.

To insert the RAG into my flow, I used langchain which allows you to very simply set up your RAG and to be able to draw from it for each request that will be made to your LLM.

Thanks to all this work, today I am proud to have publications that seem authentic and human, it is a long work and we can always improve.

The road is still long and we have some avenues for improvement for the curious:

• try Claude's model to see if we have better feedback
• do a fine tuning with several hundred LinkedIn templates to respond to a maximum of specific use cases and which will be more adapted

If you want to try and see what it looks like: lumeo.me