Large Language Model’s don’t really have a database to query on. They analyse petabytes of data on a range of topics (as part of “training”), and measure the distance between clusters of words or pixels that are most likely to appear together. They then match the pattern to a given natural language prompt, and generate a remixed output in the form of text or image. It is generally a learnt synthesis of the kind of content or image that has been created before on the subject of the prompt.
The output depends on what you ask (hence prompt engineering), but in most cases a basic human language query is enough for the system to match patterns and return some text or image which looks good and reads ok.
Question though, is novelty aside, what do you ask a system that (apparently) can answer anything? Where does it truly help beyond writing content or creating images?
It’s hard to use freedom without boundaries. For a writer it’s a blank page, for a painter, a blank canvas, and for the guy next door it’s the prompt to feed in Chat GPT for a response. Sure questions and answers are very important, but how much do we actually depend on questions and answers alone to get something done end-to-end?
It is definitely part of the process, but is it, “THE” process?
I have created probably over 2000 images on MidJourney. They are very cool and improving everyday. I love them. I have also tested various LLM bots with regards to prompt to text and text to speech. But after a point I feel this general lack of interest or ownership with respect to the generated output.
The lack of effort is nice while researching, but quite distancing when you debate publishing.
Anyone can use an LLM to write or generate images. The differentiating factor is extremely low. Building a unique voice is simply impossible. It’s more like a thing to satisfy curiosity, find quick answers, kill time and aggregate quick research. There is also a certain kind of boredom. How many pretty pictures do I need in a day? Or a month ? Or a year?
When every picture is amazing, then what is the value of amazing?
How many copy pasted listicle blogs could I post that would push people to think? To engage? Or build a community?
In short, what is their most pertinent use case in everyday life, long term? Do they stay as assistants to write, create images or spit out code? What DO I ask them to create beyond that or is that the end of human creativity ? Because they can create “apparently anything”, but none of them completely usable end-to-end, from tool to delivery in one prompt (or many).
Sure there are many “ideas from AI experts on the internet”
Let’s look at two random examples.
One such idea is using AI generated images to publish children’s comic books
Ok, but for a comic to be really successful, it is the story , narrative, characterisation and humour that has to stick. Of course Chat GPT could give the story, MidJourney could give the images. some other AI tool could probably marry it all together into a printable eBook and publish in on Amazon. But what would that comic be? A story based off concepts and patterns matched to the prompts given? LLMs could also probably rewrite paragraphs with some words that make them sound funny, but that would not create humorous storytelling.
Another idea is writing a book or a learning course using AI to curate the content
Can be done, but for a book or learning course to be valuable, it would need a great narrative, great distribution, great word of mouth and great writing that people find refreshing. Insights that they probably would not get anywhere else. LLMs are great at answers. Insights too. But would these insights be new when everyone can arrive at the same using the same LLMs? Would these answers be refreshing? Would we read them and go wow, the way that was written, it made me think? We definitely do go wow at the fact that it can write factually 90% correct things, but there is a lot more to a book, or a course than facts and correct grammar. Wikipedia also gives great answers, but how much do we use Wikipedia to write a great book?
The thing is, there is still some disconnect. The images are awesome, but not really usable for any serious origination or creation (for the most parts). The presentations give a form, but can’t be used in lets say a real life business QBR. The content is ok, factually passable between 80 to 92 %, but not really worth publishing if you truly want to become a writer and create an audience.
It does help in some research though, answering questions, summarising , paraphrasing and in creating mock up of ideas which can help the process. But it doesn’t replicate the entire creative and production process. Now let’s say that LLMs improve massively and do create the content, image or analysis end-to-end. Even then that would maybe constitute 50% of the job. The rest would be around promotion, building a distribution, creating an audience, getting a buyer and making a sale.
LLMs do know how to create distribution, but they would not be able to actually execute one. The truth is that many of us know “how to create distribution”. Few actually can create it.
So it brings me back to question. What do I ask it to create?
We solved for logical query resolution in Machine Learning, by converting it into a statistical problem. Instead of giving the system a series of logical tests to differentiate between a dog and an astronaut, we reverse engineered. By showing the system millions of pictures of dogs and millions of pictures of space suits, we enabled the system to pattern match and then generate a remixed concept of a dog in a space suit.
Statistical probability, converted into pixels diffused into our canvas.
The system did not try solving for logic because it’s hard to explain logical query sets that define a dog in an astronaut suit. The system used pattern matching and subsequent pattern generation based on a large data sample set to remix what a dog in a spacesuit could look like.
That is incredibly cool. But when it comes to using this ability to actually generating let’s say the space suit we might need for Mars (hypothetically since we haven’t yet gone to Mars, and hence don’t know what would be needed), what would the system create?
Would it originate based on non existing data? Origination happens when you break the pattern. What would be the score then?
In music, art, literature and science, innovative creation has only happened when a pattern was broken. But when there is no data to establish patterns, what would a system delivering responses based on probability and pattern matching, throw up?
What would be the prompt?
Also when thousands of LLM’s, all trained near-perfect, could answer similarly on the same common crawl data from the web, what would be the use case for each ? Would each LLM specialise on one thing? What would be their market share ? Since all LLMS would use the same data as a baseline, all of them theoretically would be able to answer “anything.”
Which one would we choose and how ? Would it be based on pricing or quality? Or a bit of both?
The speed of improvement in LLM’s is astonishing. So, quality would become a zero sum game after a while. In such a situation, why would one use Claude over let’s say ChatGpt? Or would they basically become the same with the exact same capabilities, just a different brand name?
Maybe the answer is that some LLMs would promise less “hallucinations” over the others?
Well that too in time would be fixed with plug-ins and weights that balance models with the right data parameters. A Wolfram Alpha LLM plug-in for mathematics, a TripAdvisor LLM plug-in for travel and so on which is already happening. Whether the TripAdvisor given trip itinerary is what you want is debatable, but it’s possible.
Now, if quality is removed as a differentiator, what would remain is price. So would that indicate a race to the bottom for the lowest? Then what would happen to market cap, monopoly and hegemony because, let’s be honest, there would be no sustainable trillion dollar valuation without near perfect monopoly.
If there is no value differentiator and thereby no moat, how will LLM A versus LLM B define the ticker price in the long term, just on their own, without attaching itself to a set of other value services where there is already an existing monopoly?
They would definitely make the process of creating a few things a lot easier. Hence cheaper. So what would we then do with the extra money and time ? Would we create less or more?
History says, we always regress to creating more. A mixture of aspiration, need to generate more income and general human curiosity. Horse drawn carriages ended, but travel and transportation exploded. Photoshop came and killed the billboard painter, but design and art expanded, with barriers being lowered. More design jobs got created. than the design jobs lost over time, even though it’s equally true, that the billboard painter never got his job back.
On the contrary we might chose to create less, but creating less could cause a challenge. Because that would pull back the economy which would lead to lesser spends, which would lead to less funding available for these systems to keep doing their computing. A system is only useful if more and more people use it, but why would more people use it if the use cases don’t expand and the need to do more things disappears as we end up doing less or the jobs get eradicated?
There is a belief that natural language prompting is the answer to LLM productivity and usage. Some say it’s AGI. But AGI requires fundamental logical reasoning and for now LLMs can only do pattern matching and pattern generation. We can’t comment on what AGI can do until we see AGI. But would an LLM as it is today, start creating a painting inspired by its own thoughts without a prompt? Or write an entire book? Maybe, if they are programmed to write a book on a set of topics every few hours.
But, when it comes to originality without pattern, the output of LLMs is kind of unknown. Sure, a good prompt can get a better output, but mostly it’s for point in time problems that don’t have scale. It could make my image better, but there is no guarantee that there will be a demand for my better image, because literally everyone would recreate the same quality using the same tool.
Instead the answer is more believable if we look at LLM tools which give us a series of recipes to try. Basically features, or use cases but also boundaries for us to play within. They take away the blank canvas. Today, most of these use the same baseline data, and work as thin wrappers that as such don’t have much value, since the core job can be done by the LLM itself.
But what happens when there is deep specialisation using proprietary data not available in a common crawl. Then each such wrapper or thin client has a monetisation and scale angle for that industry, but only for a specific set of tasks.
Unbundling has great power in stripping off pieces of value from a large solution, perfecting it and then reselling it back as a value add. You can see a gazillion examples all around. If you add the power of LLMs to existing products or networks and create further unbundles of net new solutions, that can create some economic opportunities.
So, what could be those wrappers? Or, entire new set of unbundled products? The answer probably lies in what would be the best use cases of pattern matching or pattern generation based on available data.
However, data alone cannot be the differentiator. Data on its own is not valuable. It’s the network effect of it, which is. It’s the signal of what people do with it that matters. Once the legality around licensing of some of this usage data is ironed out, businesses would be happy to use their own LLMs or licensed LLMs to monetise it and the first movers would probably gain market share.
But then there would be more incumbents, more licensing deals, more solutions, and more competition. A nesting ground for the expansion of the kind of jobs which we don’t know of as of today, but which will probably come in the future.
In the present landscape, LLMs have a ton of consumer users, but most businesses still have no idea what to do with them or how to use them, to sell more services.
Of course they can remain as a consumer tool for search, research, content, code , image creation and more. But once open source and closed source LLMs are near perfect in their output and have to compete with indigenous and proprietary LLMs trained on proprietary data, the consumer market will be far more compelling and distributed.
So, the more I think about LLMs (as they stand today), I feel like they have this capability of becoming wonderful data operating systems. Baked into the business software suite of organisations, hosted on a private cloud , trained on proprietary or public data , and able to integrate across SaaS tools using a unified API. Again, it won’t solve a lot of other complex real world problems but it will fasten many business backend processes which can help in GTM, sales, supply chain, distribution and marketing.
In the consumer space it could become the natural language layer replacing the API as we know of. What Alexa and Siri could not quite do.
We are obviously at the infancy of this and we don’t know what changes will come. Changes in robotics or AGI that can truly scale the impact of LLMs manifold. One thing is for certain though. The changes will be a lot faster and thereby carry a certain amount of friction. Jobs that we don’t know about, will be created and some jobs that we know of today, will be lost.
The elephant in the room is of course, that AGI will come over and make us all redundant.
Now a dystopian future where machines decide on their own and rule all of mankind is (maybe) possible for arguments sake, but highly unlikely. I am no expert on AI, but I take solace in human aspiration and greed. Humans always aspire for more and need a steady source of income. The unequal distribution of wealth is tolerable, as long as the mass and the middle class have the means to earn the basics and a little beyond. However, a market where the majority of people have been rendered jobless by AI, would make for a bad economy , and not much survives a bad economy.
Systems and tools become financially valuable when we can use them to improve our own economic state.