Nothing agaisnt Snakes, but did not use any
TL;DR We should use whatever it is that gets the job done with the resources at our disposal and the knowledge we think we have.
Prologue
I remember pitching Scopebrite initial idea almost one year ago to one of the current co-founders. They had access to an entire product digital agency so a certain amount of capital and human resources were somewhat guaranteed. It could be seen as something like a pre-seed. The thing is, most projects within this agency were in Next.js and not AI, whilst myself I work a lot within the ecosystem of .NET and AI.
Decisions had to be made. If Scopebrite were to leverage the available human resources of this agency, we had to use Next.js... However, in order to move fast, if I were to work only with Next.js we wouldnt even be in the beta phase we are in now. Lets keep on rocking .NET. Fast forward, we ended up deciding Next.js for the frontend and .NET Core for the backend, where I would focus on the backend, AI and product, and all other available developers would focus solely on the frontend.
For more context, Scopebrite (https://scopebrite.com/), so far, uses only Generative AI through third-party providers so its mostly prompt engineering, prompt templates, agentic reasoning with proper monitoring and observability and some retrieval augmented generation in place. There was no need for relevant yet complicated libraries like langchain or llamaindex that from my experience didnt really justify the effort given everything else. Thus, we have a full working generative AI saas product with no python at all. Most necessary tooling around prompting and other generative AI focused slices where built in-house when we need them. After all building with generative AI is still new, and in my opinion makers should figure out what really works for them. Obviouly pay attention to the most widely known libraries, but always think critically.
Prompting is not not easy. But managing prompts is harder.
First order of business when I decided to move forward with .NET Core, was to design and implement a 'good-enough' prompt management system. My requirements at the time were:
- Easy to find prompts
- Easy to change them
- Easy to add information to them
- Easy to monitor them
Which meant "do not hardcode prompts everywhere". Quite easy to start but still easier said than done. The more prompts and generative AI focused features we had i.e generate personas, generate user stories, summarize documents, etc, the more complex it became. One generative AI feature could use one to several prompts, and each needs to be managed, tracked and easily swapped in and out. I must say the custom solutions I came up are pretty reliable as of now but the work is not yet done. Time will tell if this was a good decision.
I will be looking to share more details and even screenshots in the future as our custom solution matures.
Avoiding NIH Syndrome
Having that said, we had to be open to already existing software if it does fit what we wanted to do. We experimented with some .NET libraries that did abstract way calling OpenAI and other providers, but at the end of the day most had one, two or three things that we were not happy with or they were still really early and unstable. Nonetheless, within Scopebrite we wanted a chat-based feature which would force us to extend our custom solution to really allow chat i.e keep history of conversations and allow function calling, to augment what a chat can really do. I ended up deciding that it might be relevant to go out and look again for already done solutions, and I found https://github.com/microsoft/semantic-kernel - done and maintained by Microsoft, used to build 'copilot' like chat experiences. Perfect, worth the shot and so far have only good things to say about this. Scopebrite custom solution and semantic kernel can and do coexist, each one tackles their specific use-case. Honestly, going pretty good so far.
Conclusion and Hindsight
I naturally omitted several details and some possibly interesting pain points around using SSR Next.JS with .NET Core as its backend since its not AI specific (besides it would be another full two pages, maybe next time). In hindsight, still not sure if using SSR Next.js was the best idea, but I am no expert in this yet. When it comes to using .NET Core and building custom solutions to manage prompts and invoking LLMs, could not be happier even if there is still a long road ahead, both technically speaking and in directions that Scopebrite might take. On a final note, building a product that helps building products is really fun and quite meta. You should try it, and if you would like to have early access and try Scopebrite for free, let me know. There are and will always be special conditions for indie hackers.
