Artificial Intelligence is rapidly catching on and for all good reason. With the introduction of large language models such as GPT, BERT and LLaMA, almost all industries including healthcare, finance, e-commerce and media are using these models for tasks such as Natural Language Understanding (NLU), Natural Language Generation (NLG), question answering, scheduling, information retrieval, and so on. The very popular ChatGPT, which has been in the headlines since its release, was built with GPT 3.5 and GPT 4s transformer technology.
These human-mimicking AI systems are heavily dependent on the development of agents capable of exhibiting human-like problem-solving capabilities. The three main approaches for developing agents capable of tackling complex interactive reasoning tasks are Deep Reinforcement Learning (RL), which involves training agents through a process of trial and error, Behavior Cloning (BC) through Sequence-to -Sequence Learning (seq2seq) Learning involving training agents by imitating the behavior of expert agents and LLM Prompting where LLM prompting-based generative agents produce reasonable plans and actions for complex tasks.
The RL and seq2seq-based BC approaches have some limitations, such as task decomposition, inability to maintain long-term memory, generalization to unknown tasks, and exception handling. Due to repeated LLM inference at each time step, even the above approaches are computationally expensive.
Recently, a framework called SWIFTSAGE has been proposed to address these challenges and enable agents to mimic how humans solve complex, open-ended tasks. SWIFTSAGE aims to integrate the strengths of behavior cloning and urge LLMs to improve task completion performance in complex interactive tasks. The picture draws inspiration from dual process theory, which suggests that human cognition involves two distinct systems: System 1 and System 2. System 1 involves rapid, intuitive, and automatic thinking, while System 2 involves methodical thought processes , analytical and deliberate.
The SWIFTSAGE framework consists of two modules, the SWIFT module and the SAGE module. Similar to System 1, the SWIFT module represents quick and intuitive thinking. It is implemented as a compact encoder-decoder language model that has been tuned to the action trajectories of an Oracle agent. The SWIFT module encodes short-term memory components such as previous actions, observations, places visited and the state of the current environment, followed by decoding of the next individual action, thus aiming to simulate the rapid and instinctive decision making shown by human beings.
The SAGE module, on the other hand, mimics System 2-like thought processes and uses LLMs such as GPT-4 for secondary goal planning and grounding. In the planning phase, LLMs are asked to identify the necessary elements, plan, track sub-objectives and detect and correct potential errors, while in the grounding phase, LLMs are employed to transform the derived output sub-objectives from the planning phase into a sequence of executable actions.
The SWIFT and SAGE modules have been integrated through a heuristic algorithm that determines when to activate or deactivate the SAGE module and how to combine the outputs of both modules using an action buffer mechanism. Unlike previous methods that only generate immediate next action, SWIFTSAGE engages in long-term action planning.
To evaluate the performance of SWIFTSAGE, experiments were conducted on 30 tasks from the ScienceWorld benchmark. The results demonstrated that SWIFTSAGE significantly outperforms other existing methods, such as SayCan, ReAct and Reflexion. Achieves high scores and demonstrates superior effectiveness in solving complex real-world tasks.
In conclusion, SWIFTSAGE is a promising framework that combines the strengths of behavior cloning and the LLM prompt. It can therefore be really useful for improving action planning and improving performance in complex reasoning tasks.
Check out ThePaper, Github link, and Project page.Don’t forget to subscribeour 22k+ ML SubReddit,Discord channel,ANDEmail newsletter, where we share the latest news on AI research, cool AI projects, and more. If you have any questions regarding the above article or if you have missed anything, please do not hesitate to email us atAsif@marktechpost.com
Check out 100s AI Tools in the AI Tools Club
Tanya Malhotra is a final year student at Petroleum and Energy University, Dehradun pursuing BTech in Computer Engineering with a major in Artificial Intelligence and Machine Learning.
She is a Data Science enthusiast with good analytical and critical thinking, coupled with a burning interest in acquiring new skills, leading teams, and managing work in an organized manner.
#maximize #planning #capacity #LLMs #reducing #computational #costs #Discover #SwiftSage #generative #agent #complex #interactive #reasoning #tasks #inspired #dual #process #theory #human #cognition