With respect to software development in particular, different questions often get muddled up. I realized this during a spirited discussion with a friend. These questions are:

• Will software as an artefact remain valuable?
• Will software development continue to be an economically valuable professional trade/craft?
• Will businesses where software is the primary differentiator, not physical hardware or operations, remain valuable?

These are all related but looking at them separately from first principles helps me clear my head.

Software as an artefact

This one’s easy. Software is not going away. The hosts (think R2-D2 or Hal) or interfaces (e.g. voice) will change dramatically but consumption of software should continue to trend up. By consumption, I don’t necessarily mean us staring at screens.

Optimistically, the increased usage can take the form of proactive assistants, background agents or robots making lives easier across economic levels. We will need new interfaces, delivery mechanisms and economic models. The evolution will be rapid and humanity’s going to ask for better software.

As software becomes cheaper to produce, we build more of it. Something that was not worth automating before can be automated. Irrespective of who builds it or who makes money from it, software should become more abundant.

Software development jobs

The question is not whether software development remains useful. It obviously will. The question is whether it remains economically valuable as a primary skill for as many individual developers as it is today. My current hunch is no.

The last few months have been a whirlwind. Meaningful parts of software development have been automated across companies of all sizes. The craft is evolving even if the opinions about the direction differ. Agents can write code, navigate repositories, debug problems and increasingly execute longer-running tasks.

You still need human developers to call the shots, with or without an army of agents, to build and operate meaningful production software in 2026. Theoretically, far fewer people can generate the same amount of output compared to a couple of years back though.

AI also makes software development capability accessible to people who would never have become professional programmers. Provided a foundational platform exists, end users should be able to direct their agents to customize software they own or subscribe to.

The economic benefits of software development may accrue to a smaller group of people tackling problems where technical judgment and depth remain scarce. The ability to build software itself diffuses across a much larger group as a secondary skill.

Software companies

The recent downturn where traditional software companies lost market value, poetically termed SaaSocalypse, led many to believe that software businesses* won’t be valuable in future.

I don’t believe this. Software companies can and will generate significant market value in the future. Increased software consumption helps but that is not enough. As building software gets easier, the ability to create software itself becomes less scarce. Most of a company’s alpha will have to come from what surrounds the software. This means distribution, proprietary data, workflow ownership, customer relationships, trust or deep expertise in a particular problem.

There’s also the matter of focus. Companies with heavy hardware or operations focus will still benefit from vendors doing the heavy-lifting on the software front instead of building everything in-house.

AI-driven changes could make software companies easier to start but harder to defend on the strength of good software products alone. I still expect significant market value to be created by software companies but the number of highly valuable ones may not grow in proportion to the amount of software being created and consumed.

*By “software businesses”, I mean companies whose primary differentiator is software services or products as opposed to hardware or operations. This definition is fuzzy but bear with me for the sake of simplicity.

Road ahead

The above points are about the future 4-5 years from now. We are likely to consume more software while needing less human effort to produce it. But more software does not automatically imply more software developers or more economic benefits for software companies.

If you are someone who loves software development, feel free to ignore the above and continue to focus on your craft. The world needs good software more than ever even if it doesn’t need as many developers.

Predictions regarding AI are as good as the roll of a dice at this point. However, a mental model is helpful as an individual. Feel free to poke holes in mine.

Thanks to Ankita for reading the draft.

• Can we do this?
• How hard is this?
• How long will this take?
• What is the best way to solve this?

These questions touch on different aspects of risks (value, usability, feasibility, viability) that come up when building any product. If you have been building software long enough, you realize the importance of these questions. Scoping is one way to answer them well.

AI can make software development dramatically faster but that doesn’t remove the need for judgment. If anything, this makes good scoping, i.e. deciding what and how to build, more important. Before getting into the principles for scoping well, let me clarify what I mean by the term.

Scoping, not estimation

By scoping, I mean the craft of going from a technical problem statement or business requirement to a precise proposed solution. This is different from pure time estimation where you come up with a timeline number in days/weeks/months for a particular solution to a problem. Scoping includes elements of estimation but focuses more on the validity and context of the problem statement in addition to coming up with an eventual solution.

Much has been written about the failure to estimate software work. I agree with those opinions. This doesn’t mean we should give up and never provide timelines. Instead, our goal should be to scope a problem well and often work our way backwards from a time budget instead of providing unreliable time estimates upfront.

Scoping principles

Despite advances in AI, you still have limits. They can be in terms of tokens, mental bandwidth or infrastructure cost in addition to time. There is still no free lunch. In fact, as the cost of building falls, these constraints become more important.

Below, I am sharing pointers that helped me and my teams consistently deliver production software with limited resources and often with ambiguity.

• Understand the why
• Sometimes, saying no is the right thing
• Experience, don’t just show or tell
• Address the unknown
• Start with a time budget, not the other way round

1. Understand the why

Everything happens for a reason. Every problem statement or request that comes your way has a why behind it. That doesn’t mean the why may be justified or worth your team’s resources and time. Even in cases where the problem statement warrants work, the understanding may be incomplete. As someone tasked with building a solution, you should get as good an understanding as possible.

Depending on the stage, culture and domain of your company, this can involve anything from asking a lot of questions and reading documents to directly engaging with a customer.

For example, you may be asked to build integrations with Google Drive to ingest PDF/CSV as data sources for your customers in a month. This sounds like a straightforward integration task.

But the real question is whether customers need a unified searchable repository or simply better access to their raw files. You should aim to understand:

• what building the feature will solve from a business point of view
• what triggered this requirement from an end user
• where the timeline originated

The end goal can range from closing a couple of large enterprise deals to solving a repeated ask from customers dealing with a variety of data sources. Knowing the why will help you better define the scope of your solution.

Sometimes, you need to build with incomplete information. In those cases, your scope can include the need to gather more information without putting in a large amount of resources.

In most early stage startups, finding PMF is the single most important thing. To do that, sometimes you should just build that feature. But your decision-making and implementation can be much better if you know why something is needed at a particular date.

2. Sometimes, saying no is the right thing

Saying no is the simplest thing to do in theory. It may be the hardest point to practice well from this list. You can’t just shoot down something because it is hard or agree because it is easy.

Once you understand why something needs to be built, the next question is whether it is worth building at all. Your decision should be based on relevant trade-offs: technical, business and/or cost to name a few.

Even if AI agents let you build a feature 10 times as easily, it may negatively affect an end-user’s experience or increase your infrastructure cost. It can also waste all kinds of resources from your team that could’ve been put to better use elsewhere - tokens, focus, sales collateral, marketing budget.

For example, a feature may take an AI agent a few hours to build but require significant infrastructure work to satisfy an enterprise customer’s requirement around data residency.

There are many times when you should not hesitate or overthink. For example, at a pre-PMF startup, moving fast can give you a high alpha if you learn from a mistake. I am not talking about those situations as much because in my experience, builders are more prone to saying yes than no. 🙂

3. Experience, don’t just show or tell

With AI agents, it has become ridiculously easy to build a minimal usable version of a small app or a feature. When building becomes this cheap, sitting in a crude version of the user’s seat can be more valuable than discussing the idea abstractly. This can lead to better brainstorming, feasibility evaluations and more. There is no excuse to not take this approach now.

For example, a couple of years ago, instead of debating whether our users would want a natural language search, I built a crude version, put our own research data into it and asked our team to try it out. We were then able to build conviction, quickly iterate with internal feedback and finalize its scope in a few days.

4. Address the unknown

The previous point was about reducing uncertainty by simulating user experience. This one is about identifying the potential implementation details that you don’t yet understand.

Many software projects get delayed because of uncertain aspects. This can include a new API, a marketplace listing process or a new framework. That is why simply breaking down a problem statement into smaller chunks is not enough. You need to identify and isolate the most uncertain parts, and systematically address them before anything else.

As a rule of thumb, third-party dependencies are almost never a hoot. Even if you plan to use an industry-standard tool, marketplace or API, you better familiarize yourself quickly. If there is an approval process involved, go take a look at forums and independent experiences, not just the official docs. (Yes, I have been burnt in the past.)

AI can now help you reduce the uncertainties quicker than manual research or building. For example, create deep research reports, build working prototypes or, if token limits allow, ask your agent to do whatever it takes (with a clearly specified goal of course). I recommend that you RTFM but you can cover a lot of ground beforehand or in parallel.

5. Start with a time budget, not the other way round

We had started by talking about the failure of software estimation. That being said, you can’t just wave your hand and work without any timelines.

If something is worth doing, you should decide how much time you are willing to spend on it. If you have collaborators or stakeholders, align on a time budget for a project. If you are working solo, practice timeboxing.

This need not be set in stone on day 1 either. As your understanding of the reasons, goals, and unknowns improve, you may improvise on either the scope or the timeline.

xkcd #1425, "Tasks"

Wrapping up

My point of view on this topic has been shaped by my experiences at early-stage, fast-moving and growing startups with no appetite for bureaucracy. If you work at a large enough company, e.g. FAANG or a Fortune 500 enterprise, there will be additional considerations that I won’t get into. You can look at this blog for some perspective.

However, considering the current AI-charged environment, I hope even large companies will continue to focus more on the craft of building software and less on the bureaucratic side.

AI hasn’t made scoping obsolete. It has changed what good scoping looks like. When building and experimentation become cheaper, judgment becomes precious. This includes understanding why a problem matters, deciding what not to build, setting constraints, experiencing the solution and identifying uncertainty early.

You can build more than ever before. Good scoping helps you decide what to build and do it well.

Resources

• How to scope work
• How I estimate work
• Software estimates have never worked and never will
• Why software projects take longer than you think: a statistical model

Thanks to Ankita, Ashutosh and Aayush for reading the draft of this.

This is a quick story, not so much a tutorial, of how I got what I wanted using Strava and my coding agent. Here is what the final infographic looked like:

The fun part to getting to the above was that Strava’s numbers alone could not really tell this story.

Data, gym board photos and a pipeline

I use Samsung Fit 3 to track my workouts and the Strava mobile app to track my runs. The fitness tracker data syncs with Strava app on a periodic basis. This means Strava has the obvious metadata: activity type (workout/run, distance, duration, elevation, dates).

But here’s the interesting bit. For most of my gym sessions, the workout routines are in photos I attach to my posts. They were photographs of the day’s workout routine from my gym’s board. Lazy, I know. The board contains the actual session structure: movements, focus area, and the difference between a strength day, a mobility day, and something in between.

Strava only knows that I did a workout. So part of the project became a small image extraction and parsing pipeline. The core idea is to combine Strava’s structured data with the image-based context I had been casually attaching to posts.

Fetch Strava activities + photos -> local cache -> agent reads and processes gym-board photos -> JSON-based description -> HTML infographic -> PNG poster

The above pipeline consists of a couple of simple Python scripts. If some photos were already downloaded, the relevant script skipped them. If an image had already been processed, it was not processed again. If one extracted workout looked wrong, I could delete just that JSON file and regenerate it.

Using coding agent for the vision task

Claude Code’s image-reading ability acted as a convenient vision layer. A script downloads the Strava photos and then prints the list of images that still need an associated JSON. For each relevant gym-board photo, the agent writes a small JSON file describing the workout.

Something like:

{
  "type": "workout",
  "exercises": [
    {
      "name": "Goblet squat",
      "sets": 4,
      "reps": "8-10",
      "weight": "24kg",
      "muscle_groups": ["legs"]
    }
  ],
  "notes": "optional"
}

For a personal project, this agent-in-the-loop workflow was fast and easy. Once a JSON description exists, that image is considered processed. This made the vision step resumable and easy to fix.

A little classification goes a long way

My Strava activity labels were not perfectly consistent. Some strength sessions were tagged as “weight training” and others as generic “workouts”. Mobility sessions also sometimes showed up as “workouts”.

So the code uses a mix of Strava activity type, activity title, and the extracted photo data to decide whether something is strength, mobility, running, swimming, yoga, or something else. That was enough to make the final poster feel accurate without turning this into a more complex classification project.

Trying out Strava MCP

I briefly tried using the recently released Strava MCP server to see if it could replace the scripts. It could have been useful if we only required activity metadata, but not for this project since there was no way to fetch the photos.

To sum up

This was a nice example of AI helping me do something I wouldn’t have had the time to do earlier. You can find or build on top of the code here: analyticalmonk/strava-reporting. The easiest way to build on it is probably to clone the repo, point your coding agent at the README, update the config, and let it help you shape the report around your own Strava data.

For me, the output was an infographic. For you, it could be a running year-in-review, a cycling summary, a race-prep log, or just a better-looking personal fitness recap.

I may set up a personalized fitness agent with local models next. Meanwhile, if you build on top of this project or have a fitness/health-related agent workflow, I’ll love to hear about it!

Thanks to Ankita for reading the draft.

The question feels more important to me now as coding agents have taken off. It is tempting to treat debugging as a tedious software development task that will be automated. But software engineers who have spent time in production trenches can recall not only frustration caused by bugs but also the feeling of exhilaration after successfully debugging and fixing a tricky issue.

Effective debugging is a mix of hypothesis construction, evidence gathering and rigorous examination. Sometimes the answer can be handed to you on a platter by your observability stack, logs or an agent, but often it requires a mix of intuition and skillful decision making.

I love coding agents and I’m bullish on AI for knowledge work. But there is a particular type of failure I have run into over the last few months when debugging a reasonably large pre-LLM codebase. An agent can produce a confident, detailed, and wrong explanation. This wrong answer can be accompanied with strong evidence.

For example, I was investigating a failure, and a coding agent blamed it on a recent code change, even though the behaviour had existed before the diff. Another time, the agent’s proposed explanation could not have occurred because of the application’s schema.

The defense against such agent-side failures is the same discipline good engineers already apply. When something looks like a regression, questions worth asking are:

• Was this regression actually introduced by the code/change being looked into, or does it have a different origin?
• Is this code path even triggered in the scenario I’m worried about?
• Does the data model permit the failure being described?

Many confident-but-wrong findings collapse the moment you force them through such questions. Otherwise, an agent can happily attribute long-standing behavior to a new diff, or invent a failure mode the data model doesn’t allow.
Scaffolding around agents can help: a structured review process, good context retrieval, schema information, targeted searches to validate assumptions. But scaffolding only narrows the gap between the possible and correct explanation. For now, that gap often needs a human in the loop.

This lesson matters even more for those new to software development. Agents can confidently develop theories and at times lose confidence about their own results in the face of questioning as well. If developers rely heavily on agents without developing enough muscle for understanding code and failure modes, they may not be prepared to judge an agent’s output in critical situations.

So how does one get better at debugging? I don’t think there is a repeatable formula but here are some pointers that can help.

• First, develop a habit of starting from the symptom and working backwards. Articulate the failure. Identify the starting point and recent changes. Track parts of the system that may be responsible.
• Second, make your hypotheses explicit. Don’t just say “maybe it’s the cache” or “maybe the query is wrong.” Write down what would have to be true for that explanation to work, and then look for evidence that confirms or falsifies it.
• Third, learn the invariants of the systems you work on. Which fields are always present? Which jobs are idempotent? Which paths are impossible? Which services are eventually consistent? A surprising amount of debugging comes down to knowing which possible explanations cannot actually happen.
• Finally, when using agents, don’t only ask them for answers. Ask them to show their debugging path. Ask about the assumptions being made, and evidence supporting any claim. You are responsible for verification of the agent’s hypothesis as well.

Yes, you should have test suites, observability and great contextual retrieval pipelines for agents. But production codebases, old or new, aren’t perfect. They have their quirks, technical debt and aren’t being overhauled overnight.
That is why the craft of debugging remains important. As someone building software, you should actively nurture it.

Thanks to Ankita, Ashutosh and Raghav for reading the draft of this.

Context is important. This post is about my experience as a tech startup co-founder for the past 4 years. I spent my late 20s with the former as my primary identity, for better or worse.

I am 31 years old, and based out of India. I have been a misfit since much of my adolescence in most social situations. I had a middle class upbringing and spent much of my childhood in tier-2 cities. I am fascinated by technology, especially AI, and its impact on the human experience.

My startup, Looppanel, operates remotely and was started in late 2021 during the pandemic and is VC-backed. Along with an amazing team, I am responsible for ensuring that we solve our customer’s problems by building the best possible product using the best available technology. We serve customers in more than a dozen countries in 6 continents including large enterprises and some names that you would have heard of.

Overview

The startup experience is interesting because you don’t just learn new things, you get to dig deeper into the meaning of lessons you already knew.

Three big learnings in my journey are related to time. They are:

• Timing matters
• Accept the past
• Good things take time

Over the last few years, I faced situations with an intensity and frequency which instilled a deep appreciation of the above principles.

Timing matters

The most tactical and frequent way I have seen the importance of timing play out at Looppanel is when prioritising product decisions. We frequently make decisions, right and wrong, to deal with unexpected occurrences such as sales requirements or taking advantage of a market tailwind. The scale of the relevant time period in such situations can vary from hours to years.

If you do something a week later than it was supposed to be done, it may turn out to be meaningless. If you are too early to market, you will struggle. We were early when we introduced our AI features and struggled with customer scepticism. As AI became mainstream, our approach gained wide acceptance quickly.

This principle plays out in personal life too. A difference in personal situations compared to professional life is you can’t and maybe shouldn’t rely on logic as much. Your gut/heart can often give a better answer than cold reasoning would. This can mean something as simple as returning a friend’s call when they need you for a few minutes instead of waiting until the weekend.

Shifting back to work, Looppanel benefited from starting up during a global lockdown. Our product’s first version was a meeting assistant and analysis tool for user research calls. We were able to get our first customer even before I wrote any code.
Related: Why now?

On the flip side, I am also aware of multiple startups that started before the 2020 pandemic with a similar idea and struggled simply because not enough research sessions were being conducted remotely. This leads me to my second learning about time.

Accept the past

You may understand the importance of timing and take decisions to the best of your ability. You will still often be wrong and plans won’t work out. For example, when starting Looppanel, we focused on building the best possible product for a significant time period at the expense of monetization. There’s no substitute for building a great product early on but we could have revised our business goals earlier than we did.

There have also been times when I made a decision informed by emotion instead of objectivity. I can recall more instances from work where I wish I’d acted differently. Such wishful thinking isn’t productive though. It’s great to learn from the past but obsessing over it is wasteful at best and destructive at worst.

In hindsight, I should have managed my personal relationships differently over the last few years as well. I became comfortably distant from a lot of friends and family members. As an introverted person, it didn’t help that I spent much of the pandemic starting a startup and writing an O’Reilly book. All I can do now is to mend relationships to the best of my abilities and look forward.

A habit that’s helped me immensely with managing my emotions about the past is journaling. You can’t always shut down how you feel but writing about it can be therapeutic.

Good things take time

We have all heard this since childhood. It’s hard to live it though. This is especially true in the startup world where ‘overnight’ successes and rapid growth are celebrated with an almost religious fervour.

I distinctly recall an entrepreneurship lecture at my college where a business owner had mentioned that if your company survives its first 3 years, the chances of becoming successful become disproportionately high compared to the average. More recently, I recall Michael Seibel from YCombinator mentioning how the period from 18 to 36 months is the toughest for most first-time startup founders, including ones which eventually succeed. I have experienced this first-hand in the last 3 years.

2024 has been the most successful year of Looppanel by a distance. Much of our success is built on the foundation we laid for ourselves as individuals, a team and a company previously. This includes learning from the failures we endured. Even then, it was harder to envision, if not hope or believe, this success at the beginning of last year.

At a personal level, it would’ve helped immensely had I been half as good at thinking about building products and empathizing with users, not just application of technology, when we’d started out. However, I needed time and experience to become better.

Summing up

While hard work is often overrated in the professional sphere, persistence or grit is underrated, particularly in startups. The best results (and stories) emerge not just from working hard, but from staying in the game long enough and remaining mindful of timing.
Note: Knowing when to give up is equally important but more on that later.

The startup journey has taught me about understanding when to move, accepting what you can’t change, and having the grit to persist when immediate results aren’t visible. This is true as much for building a company as it’s for living your life.

The feature I’m referencing - AI notes - generates high-quality notes on top of user interview transcripts. Many of our users tried out similar tools but told us over and over that our version was superior.

To someone who’s not analyzing user research data, these notes will not appear to be any different than those from a generic AI meeting assistant.

When our user looks at these notes, there are 2 aspects that they often evaluate them on - does the note capture a relevant part of the interview (chunking), and does the note describe the part correctly and objectively (summarization)? These 2 aspects are what we will dive a little deeper into.

Specifically, we will talk about 2 components of our AI notes pipeline - chunking and summarization - and lessons learnt while building them. These components also highlight how crucial data transformation/engineering and user experience (UX) are for an LLM-based feature in addition to the underlying model.

Note: please keep in mind that a lot more work went into making a successful feature in addition to the 2 components being discussed. They stand out from a technical and product POV but are only pieces of a bigger puzzle which involves user research, design and software development. In addition, I am not sharing the specific technical implementation but a high level overview and lessons. I hope that you will be able to apply them to your specific situation if relevant.

Chunking

Search for ‘chunking text for LLMs’ and you will come across many articles and documentation pages. They describe multiple methods to break down text documents into smaller parts (chunks). Chunking is mostly talked about in the context of RAG or search.

In our case, we were doing a version of text mining. We wanted to break down a user interview transcript into multiple chunks. Each chunk needed to make sense as a semantic unit in context of the entire interview. It was early 2023 and we were severely restricted by context length and semantic boundaries.

Note: 4k context length limit sounds like a distant memory compared to the 100k+ limits that have become the norm for large language models now.

Semantic chunking sounded cool as a concept but out-of-the-box implementations didn’t satisfy our use case, same as all of the other techniques on offer. For example, if you simply broke down a conversation based on number of sentences or even semantic similarity, there was a high probability that the natural flow of the conversation would not be captured. This would result in the end user spending signficant cognitive energy reviewing the chunks and the corresponding notes.

With user context and use case in mind, we wrote a chunking implementation that specifically worked for user interviews. It accounted for distinction between interviewer and interviewee speech, and identified topic changes in the conversation, to find relevant boundaries. This was an example of a domain-specific chunking methodology.

The output from this method made a lot of sense for the users and their relevance contributed to not just AI notes but multiple features that we built afterwards.

Summarization

Summaries of text documents are seen as a commodity at this point. I am fascinated by how the term can take different meanings based on the use case and who they are meant for. For us, chunking was technically challenging but generating the summary text turned out to be one of the most nuanced parts from a user experience perspective.

In addition to correctness, a user can care deeply about the tone, length, and choice of words of a summary. If you don’t get this right, the user will not care about all the fancy data engineering or modelling work you may have done until that point.

This is something we learnt after releasing a beta version. The beta users would fixate on the language of notes and ignore everything else. We iterated multiple times on relevant prompts used to create the summary by experimenting with the length, tone, etc.

In hindsight, I would’ve leveraged domain expertise earlier and a lot more to finalize the prompts used to generate the note text that a user read. That’s the approach we ended up taking for similar problem statements since then.

Wrapping up

LLMs are evolving rapidly and so is the ecosystem and user expectations around them. When building an LLM-powered product, technology can play an even bigger role than has been the norm in the last decade or so. Hence, it gets easier to ignore the user experience when working on an AI product more than usual. However, building something that actually works involves maintaining a fine balance between the technology and UX.

Note: As hard as it may be for you to believe, the above introduction was written by a human. Or not. It’s hard to tell.

Outline

1. Chat isn’t always the answer
2. You don’t need to know everything
   a. It doesn’t hurt to know a bit more than you need
3. Tinker
4. Open source is your friend but APIs are not your enemy
5. Demos will take a weekend, production will keep you up on weekends

Lesson 1: Chat isn’t always the answer

By now, you must know. If you don’t, you sure should be glad that you are reading this. You probably don’t need to make ChatGPT for X. Maybe you do. Most probably, you don’t.

Chat can be great because it’s flexible. For a lot of use-cases, it’s not great because it’s too flexible. Your users don’t always want to stare at a canvas of infinite possibilities. Sometimes, they need help. If they want to get their professional work done, they definitely could use some help.

What interface do they need then? I don’t know. That’s for you to figure out by talking with them. I can only provide great lessons, not answers.

Note: Here’s what we have built at Looppanel. We don’t allow our users to talk with their interview transcripts. Not yet anyway. Instead, we provide them with meaningful interview notes tied to evidence (transcript text and timestamp) and their inputs (themes and questions). Hallucination isn’t an option when your job is to find out the truth.

Lesson 2: You don’t need to know everything

There’s too much happening all the time. I am not talking about the significant issues of global warming or geopolitics. I am referring to your X (previously Twitter) feed and Arxiv.

You tried but are barely able to keep up. That’s okay. If you try too hard to drink from a firehose, there’s a chance that you will drown.

First, try to figure out what you need to know to make whatever it is you need to make.

Lesson 2a: It doesn’t hurt to know a little extra

It’s okay to not know what RAG stands for and why vector databases are the talk of VC town. However, if you have a rudimentary understanding of embeddings, that will be helpful if a potential business use-case presents itself.

Skim Andrej Karpathy and Jeremy Howard’s X feeds occasionally. Subscribe to a couple of newsletters or blogs even if you skim them once a week. Don’t try too hard to keep up but use your downtime to explore. If you are feeling adventurous, try a MOOC.

Lesson 3: Tinker

If you want to explore, it’s better to be hands-on. Talk shit with ChatGPT/Claude/Llama or port a SOTA LLM in C. The world’s your oyster. You can learn a lot by tinkering in a field moving as fast as good-old AI is.

It took the experts a while to figure out that asking an LLM to “think step by step” can make it work better. Maybe you could’ve done it had you tinkered with a model (while being polite).

Lesson 4: Open source is your friend but APIs are not your enemy

Open source is great. Use them as much as you can and give back whenever possible. But if you are on a strict timeline or working out of a VC-funded garage, picking up an API before going through the Open LLM board may make more sense.

Once you have a validated use-case and method, it can be easier to fine-tune or go all-in on open source.

Note: I can be wrong with this one and starting with OSS may be the answer. I hope to find out more as I tinker with fine-tuning and smaller models.

Lesson 5: Demos will take a weekend, production will keep you up on weekends

Sometimes, I wish it were as easy to make an LLM work in production as it is to make a mindblowing demo.
Then again, where’s the fun in that? I can’t convincingly answer because I was busy figuring out why our GPT-powered pipeline had inexplicably timed out again. :)

Test, test some more and then monitor. Implement some guardrails and checks when passing data to downstream tasks. Add a cache if that makes sense for your use-case.

Things can go wrong on multiple parameters (cost, latency, model drift) and there’s a lot to think about. Chip Huyen wrote about this in Building LLM applications for production that I recommend you read in case you haven’t.

(Bonus) Lesson 6: Have fun and be responsible

If you have the privilege of working with the fascinating technology of LLMs, remember to have fun tinkering with it. Be curious and help others understand the technology. Be responsible in its usage and encourage others to do the same.

In conclusion

It’s both a joy and occasionally terrifying to witness the shift that’s currently underway. The world has changed recently due to the breakthroughs in AI technology. It will continue to change in ways we can’t imagine right now. Let’s gear up for the ride!

Let me clarify what I mean by enterprise and user experience here.

• “Enterprise software” refers to software which are used for business use-cases. Tools like Notion, GitHub or Figma which have enterprise versions but are frequently used outside of a workplace qualify too.
• User experience refers to how a user feels when interacting with a system. It goes beyond utility. It includes the emotions of joy, empowerment and satisfaction.

When thinking about this topic, I came across a related blog post by Calvin French-Owen, Segment’s co-founder and ex-CTO. He differentiates B2B UX based on the sales motion (top-down or bottom-up). The UX will match the buyer, not the end-user. If the practitioner is the end-user, UX and aesthetics matter significantly in the buying decision. Visibility and reporting takes precedence over UX for an executive. That makes a lot of sense.
I explore adjacent theories. My opinions are loosely held and I believe there’s something interesting as well as useful to be found along this line of thought.

UX, sales motion and type of work

Building on Calvin’s idea, I hypothesise that a bottom-up approach works out better on average for software for people directly building products. Hence, software meant for them has better UX on average too.

Examples of builder roles include designers, developers or data scientists. Individual contribution with low high-level stakeholder intervention in day-to-day goal-setting is more common. Pain points that can have an outsized impact are often discovered during daily work rather than meetings.
In contrast, visibility is critical in software meant for roles such as sales or customer success. Pain points that are prioritised to be solved by tools are often aligned on in meetings involving managers and executives.

As a result, a designer or developer is likely to try and find a tool to make their lives better. Preliminary adoption and validation can be done more easily by an end-user in such cases. The bottom-up approach kicks in and UX becomes important.

How much do they care?

There may also be a correlation between how much a person cares about their work and how much they will think actively about improving the UX of their tools.

If you are creating bottom-up software, this correlation can help you qualify users. People who deeply care about their day-to-day work in addition to having a painful problem will provide you with the best possible feedback. They are more likely to have tried hacky solutions too. You can learn from them. The business implications aside, startups can be good early product users for the same reason. It will be harder to satisfy these users but you will have higher chances to make a good product that works for even the laziest of users.

Do people in certain roles tend to care more about their work on average? I don’t know but it’s an interesting question. That can relate this hypothesis with the previous one.

In conclusion

My hypotheses are based on observations and experiences. More than being definite, it can open up interesting lines of thought. No matter who you are building for, user experience is becoming more important. It will be interesting to see the direction our software experiences take with the advent of AI too. It’s an exciting time to build!

Thanks to Ankita Mathur for reading drafts of this post.

Hacker News thread for this post: https://news.ycombinator.com/item?id=36104092

This article provides a higher-level overview of the projects and underlying motivations. You may decide to pursue or replicate this path. I may decide to write about individual pieces. For now, I present a summary of my experience and learnings.

Sidenote: From a knowledge perspective, I was already comfortable with machine learning, neural networks, the PyData stack and cloud computing. Keep that in mind if you plan on replicating this approach.

Getting started

My initial list of projects was as follows:

• Project(s) with OpenAI’s GPT-2 to better understand language models
• An ML-powered chatbot with an aim to understand NLU better
• A tool to transcript and analyze audio conversations (online or phone calls)

The list was intentionally open-ended. The plan was to do projects of interest and refine ideas along the way. The choice of exploring the GPT-2 language model was because of abundance of related learning resources and discussion. Experiments around the first two set of ideas are described below. I haven’t had a chance to start working on the third one yet.

Sidenote: A week after I had started this project, OpenAI released their API. If you are taken in by the buzz, next section may be of particular interest.

Playing around with GPT-2

GPT-2 showed up during the emergence of transformer architecture and transfer learning in NLP in 2018-19. Transfer learning is a powerful concept. It allows pretrained large neural networks (language models in this case) to be finetuned for downstream tasks. This saves practitioners the trouble of retraining entire models from scratch. This flexibility is a big reason behind language models’ success.

When starting out, I searched around for existing GPT-2 projects to replicate. That is when I stumbled upon related Python packages by Max Woolf. His work makes it easy for even absolute beginners to start out. It was then that I decided to make a parody Twitter bot as a first project.

Parody Twitter bot

A parody bot of Twitter user dril had become popular in 2019. It was powered by a GPT-2 model finetuned on dril’s tweets.
In the same spirit, I made a parody bot account that was trained on a mix of 5 different parody accounts. These included personas of Queen Elizabeth, Lord Voldemort, Darth Vader and Bored Elon Musk. To get the tweets, I used the amazing twint package.

The resultant bot account is named Queen Darth Voldemort Musk, first of their name. Yep, you read that right.

The information required to replicate this project can be found here.

Transformers, BERT and GPT-2

Then, I took a little detour to read about the theory behind GPT-2 and transfer learning in NLP. BERT is another language model that kept coming up. Both GPT and BERT are based off of the transformer architecture. However,BERT has just the encoder blocks and GPT-2 has just the decoder blocks from the transformer. That makes them more suitable for different kind of tasks.
Trying to understand the technical differences and capabilities of these two popular models is a helpful exercise. I highly recommend doing that!

Here are some relevant links around this topic:

• NLP’s ImageNet moment has arrived
• Transformers in NLP - a brief survey
• Notes on the transformer architecture
• Notes on GPT-2 and BERT

This detour also led me to my next project idea.

Evaluating NLP benchmarks

Triumphs in machine learning are tied to their performance on standard benchmarks. Even if you are not a researcher, knowing about, a benchmark such as GLUE will give you important insights about the field. If you set about contrasting BERT with GPT-2 even without in-depth expertise, looking at the benchmarks they are evaluated on will let you know the problems they are suited for. That is helpful as a practitioner.

NLP benchmarks such as GLUE and SQuAD are comprised of multiple tasks. Different tasks test for proficiency in different aspects of language such as entity recognition, question-answering and coreference resolution. In combination, they test for general language understanding.

For this project, I had two goals:

• Evaluate both BERT and GPT-2 on at least one task from relevant benchmarks
• Implement various tasks from popular benchmarks

For implementation, I used the transformers Python library by HuggingFace. I will release the associated notebook soon and hopefully write more about this subtopic.

Meanwhile, here are some helpful links:

• GLUE Explained: Understanding BERT Through Benchmarks
• Video - State of the art in NLP
• Why isn’t GPT-2 on the GLUE leadership board?
• The Big Table of (NLP) Tasks by HuggingFace

Practical NLU: conversational interfaces

Conversational interfaces are becoming increasingly prevalent. They are going to be increasingly important for consumer-facing applications. This is especially true for emerging economies where voice-first interfaces can add immense value for new internet users.

As I was thinking about conversational interfaces, I came across an online event by RASA. I attended some of the talks. They helped me get an intuition from the perspective of a developer. Post that, my inclination towards voice interfaces helped me pick my next project.

AI voice assistant

This is a great tutorial - How to build a voice assistant with open source Rasa and Mozilla tools. I simply followed it end to end.

Most of the functionalities implemented in this are available as managed services. As a matter of personal choice, I chose to start off with open source tools. There are trade-offs to be considered when thinking about a production scenario, of course.

RASA has developed a rich set of resources and an active community. They are extremely helpful.

Customer service chatbot

Having worked with open source tools, I wanted to try out a managed service. Options are provided by the likes of GCP, AWS, Azure and IBM. I chose to go with Google’s Dialogflow.

Once you are familiar with fundamentals such as intents and actions, it is straightforward to build a bot. The pretrained bots are a convenient feature but importing them was not a smooth experience. The one-bot-per-project feature and the lag it took for it to show up confused me for a few minutes.

Concluding thoughts

It has been an interesting experience going about the above experiments. At times, I had an itch to go deeper into topics. Choosing to let that go allowed me to explore the breadth at hand. It also let me squeeze short hacking sessions into my schedule.

There have been some exciting developments in the natural language processing field of late. In emerging economies, voice as an interface is increasingly gaining adoption. Improved machine understanding of language will also improve democratization of technology.

At the same time, we need to be realistic about its limits. Use-cases of OpenAI’s GPT-3 powered API have gone viral. So much so that their CEO, Sam Altman, made a sarcastic tweet about the hype.
RASA has laid out a concept of 5 levels of conversational AI. They also mention that we are perhaps a decade away from the fifth level.

Language has been key to humanity’s progress. Exploring it through the lens of emerging technologies is an interesting endeavour. I hope that the train of innovation will continue in a responsible manner.
If you want to have a discussion or share thoughts, feel free to reach out.

Follow the discussion on Hacker News

Source for header image: WikiMedia

We have all heard the stories. They met, they started up and they conquered. They are the co-founders.
There are a ton of variables at play in an entrepreneurial journey. These include the market, capital and good ol’ luck. The market won’t care about whether a startup team has been awake for days or not drawn a salary. The humans you work with will. Good co-founders and founding teams can help each other manage a startup’s variability and chaos. Thinking deeply about who you want to work with is as important as anything else as an entrepreneur.

There are no strict rules or absolutes when it comes to a startup. Popular startup folklore may lead you to believe that finding a co-founder is almost entirely down to luck. Luck is a factor, yes. That doesn’t mean there is no method to the madness.
This article aims to lay down some frameworks and a philosophy to think about a co-founder and to a lesser extent, founding team. There are no easy answers to be found. But if you can ask the right questions, that can immensely help your journey.

Starting with self

Before ‘product-market’ fit comes ‘founder-market’, ‘founder-product’ and ‘founder-founder’ fit. You need to be honest with yourself before you can go about expecting the same of others. It does not come easy. It is easily worth the effort though. Ask questions such as:

• Why are you doing this startup?
• Do you really care about the problem?
• What is your financial position? How important is money to you?
• How much time are you willing to commit?
• Do you think deeply about things or want to get shit done quickly?
• What kind of work-life balance do you want?
• Are you comfortable delegating or do you prefer to micro-manage?
  • Alternatively, are you a control freak or not?

.. and so on.

I am not qualified to comment on right or wrong answers. A relevant article in that regard is Mark Andreessen’s Why not to do a startup. I can tell you this much - the clarity will help you make decisions quickly and decisively. That is a big positive.

Solo or not

“Do you even need a co-founder?”

If you plan to startup and don’t have a college dormmate co-founder, you most likely have asked the above. Paraphrasing a founder friend, “if you want to stay healthy and not wear yourself out, do not go solo.” That does make sense but there’s more to it.

Below framework helps you think about this question.

X-axis refers to the core skill that your startup requires. For example, a deep tech startup will hinge heavily on technology and engineering. An ops-heavy one requires relevant operations expertise.
Y-axis refers to your need for control or micro-management as a professional. The term “control freak” doesn’t have a negative connotation here.

I am still trying to digest this skill-control framework. It does offer some clarity though.

The dream team

This is the big question. What makes a good co-founder? There are couple of common-sense frameworks that I have recently come across which try to answer this.

Weighing the skills, ideas and values

Assess skills, ideas and values on an individual level.

• ‘Skills’ are self-explanatory.
• ‘Ideas’ refers to the way of thinking and approaching problems. An example of contrasting approaches is an instinct to get shit done quickly versus thinking deeply first.
• ‘Values’ are the principles that someone uses to make decisions, big or small. It is an abstract concept. You assess it the same way that you would possibly do it for a life-partner. It’s hard to ascertain, I know.

The above table describes the most probable scenarios that can arise as a result of the corresponding skill-idea-value dynamic between the co-founders. It is a neat way to simplify a complex equation.

Thinking about edge

Entrepreneur First (EF) is a global program that helps ambitious individuals build deep-tech startups. It helps you find a co-founder and raise capital. Their philosophy heavily uses the concept of ‘edge’. EF defines 3 broad edge types:

• Domain (D): Years of experience in a certain industry along with insights on how it can be improved
• Technical (T): Experts and researchers in a particular technology
• Catalyst (C): Business or technology generalists

EF’s thesis states that successful deep tech companies are primarily formed by relevant combinations of edges. For example, D+T or D+C(tech).

Boiling it down

In simpler words, you need complimentary skillsets and temperament on a good co-founding team. If someone is building a product, someone needs to sell it. There will be exceptions but this holds true in most cases. Another key takeaway is that the values need to be aligned from the get go. That is the most human element of the lot. It may be overlooked if you rush through things. Avoid that at all costs.

Parting words

Take all of the above with a pinch of salt. I am not a successful entrepreneur. In fact, I am looking for a co-founder these days. That is why I refrain from being prescriptive. That being said, I have tapped into enough collective wisdom and gained experience to know that the above frameworks are helpful.

Do not fret too much about the nitty-gritties. The frameworks will guide you but that is all they will do. The journey is yours to make. Try to be self-aware and honest. Enjoy along the way.

May the force be with you!

Source for header image: Pikrepo