LangChain: the trendiest web framework of 2023, thanks to artificial intelligence

LangChain is a programming framework for using large language models (LLM) in applications. Like everything in Generative AI, things moved incredibly fast for the project. It started as a Python tool in October 2022, then added TypeScript support in February. In April, it supported multiple JavaScript environments, including Node.js, Browser, Cloudflare Workers, Vercel/Next.js, Deno, and Supabase Edge Functions.

So what do JavaScript developers need to know (in particular) about LangChain and working with LLMs in general? In this post, we aim to answer that question by analyzing two recent presentations by LangChain creator Harrison Chase.

LangChain started out as an open source project, but once GitHub’s stars started piling up, it was promptly transformed into a startup. It’s been a meteoric rise for Harrison Chase, who was studying at Harvard University in 2017 but is now the CEO of one of Silicon Valley’s hottest startups. Earlier this month, Microsoft Chief Technology Officer Kevin Scott gave Chase a personal message during his Build keynote address.

Chat app All the rage

Unsurprisingly, LangChain’s primary use case currently is to build chat-based applications on top of LLM (specifically ChatGPT). As Tyler McGinnis of the popular LangChain newsletter bytes.dev wryly observed, “You can never have enough chat interfaces.”

In an interview with Charles Frye earlier this year, Chase said the best use case right now is “chatting about your docs.” LangChain offers other features to enhance the chat experience for apps, such as streaming, which in an LLM context means returning LLM token output by token, rather than all at once.

However, Chase has indicated that other interfaces will evolve rapidly.

“In the long run, there are probably better UXs than chat,” he said. “But I think it’s the immediate thing right now that you can stand up very easily, without a lot of extra work. In six months, do I expect chat to be the best UX? Probably not. But I think right now, what’s the one thing you can build right now to deliver value, that’s probably that [i.e. chat].”

Since developing applications with LLMs is such a new thing, startups like LangChain have been scrambling to find tools to help solve some of the problems with LLMs. With prompt engineering, for example, Chase indicated that it still mostly depends on the developer’s intuition as to which prompts work best. But this year, LangChain has introduced features like “tracking” to help with that.

Agents

One of LangChain’s newest features is “custom agents,” which Chase talked about at an event called LLM Bootcamp, held in San Francisco in April. He defined agents as a method of “using the language model as an engine of reasoning” to determine how to interact with the outside world based on user input.

He gave an example of interacting with a SQL database. He explained that typically you have a natural language query and a language model will convert that into a SQL query. You can run that query and return the result to the language model, ask it to synthesize against the original question, and you end up with what Chase called “this natural language wrapper around a SQL database.”

Where agents come in is the handling of what Chase called “the edge cases,” which could be (for example) a hallucinatory part of an LLM’s output at any point during the example above.

“You use the LLM which is the agent to choose a tool to use and also the input for that tool,” he explained. “You then […] you take that action, get a comment, and then feed it back into the language model. And you kind of keep doing that until a stopping condition is met.

A popular approach to agents is called “ReAct”. This has nothing to do with the popular JavaScript framework of the same name; this version of “ReAct” stands for Reason + Act. Chase said this process produces “higher quality, more reliable results” than other forms of timely engineering.

Chase admitted that “there are a lot of challenges” with the agents and that “most agents aren’t incredibly production-ready right now.”

The memory problem

Some of the problems he listed sound like basic computer science concepts, but are more challenging in the context of LLMs. For example, LLMs usually lack long-term memory. As noted in a Pinecone tutorial, “By default, LLMs are stateless, which means that every incoming query is processed independently of other interactions.”

This is one area where LangChain aims to help developers by adding components like memory into the LLM management process. In fact, in JavaScript and TypeScript, LangChain has two methods related to memory: loadMemoryVariables AND saveContext. According to the documentation, the first method “is used to fetch the data from memory (optionally using the current input values) and the second method is used to store the data in memory”.

Another form of agent that Chase talked about is Auto-GPT, a software program that allows you to configure and deploy autonomous AI agents.

“One of the things that Auto-GPT introduces is the idea of ​​long-term memory between agent and tool interactions and using a vector store retriever for that,” he said, referring to vector databases.

The new LAMP battery?

Clearly, there is still a lot to understand when it comes to building applications with the LLM. In its Build keynotes, Microsoft classified LangChain as part of the “orchestration” layer in its “Copilot technology stack” for developers. In Microsoft’s system, orchestration includes prompt engineering and what it calls “metaprompts.”

Microsoft has its own tool, Semantic Kernel, which does a similar thing to LangChain. He also announced a new tool called Prompt Flow, which Microsoft CTO Kevin Scott said is “another orchestration mechanism that actually unifies LangChain and the Semantic Kernel.”

It’s also worth noting the word “chain” in LangChain’s name, which indicates that it can interoperate with other tools, not just various LLMs, but other development frameworks as well. In May, Cloudflare announced LangChain support for its Workers framework.

A new acronym has even been coined involving LangChain: OPL, which stands for OpenAI, Pinecone and LangChain. The inspiration was probably the LAMP stack (Linux, Apache, MySQL, PHP/Perl/Python), which was a key part of the 1990s and led to the emergence of Web 2.0. Who knows if OPL will stick around as a term — and, of course, its components aren’t all open source — but regardless, it’s a good indication that LangChain is already a big part of many developers’ personal stacks.