How Planview uses Amazon Bedrock to build a scalable AI assistant for portfolio and project management

This text was co-authored with Planview’s Lee Rehwinkel.

Immediately’s companies face quite a few challenges in managing advanced initiatives and initiatives, deriving precious insights from massive quantities of information, and making well timed selections. These obstacles typically trigger productiveness bottlenecks for challenge managers and senior executives, hindering their means to successfully drive organizational success.

Planview, a number one supplier of linked work administration options, has launched an bold plan in 2023 to revolutionize the way in which its 3 million customers worldwide work together with its challenge administration purposes. To comprehend this imaginative and prescient, Planview developed an AI assistant referred to as Planview Copilot utilizing a multi-agent system powered by Amazon Bedrock.

Creating such multi-agent techniques presents a number of challenges:

• Reliably route duties to the suitable AI agent
• Entry knowledge from a wide range of sources and codecs
• Work together with a number of software APIs
• Enabling totally different product groups to create new AI abilities on their very own

To beat these challenges, Planview developed a multi-agent structure constructed utilizing Amazon Bedrock. Amazon Bedrock is a completely managed service that gives API entry to foundational fashions (FMs) from Amazon and different main AI startups. This permits builders to decide on the FM that most closely fits their use case. This method is architecturally and organizationally scalable, permitting Planview to rapidly develop and deploy new AI abilities to fulfill clients’ altering wants.

This text focuses on the primary problem: routing duties and managing a number of brokers in a generative AI structure. We explored Planview’s method to this problem in the course of the growth of Planview Copilot, sharing insights into the design selections that present environment friendly, dependable job routing.

We describe customized native brokers on this article as a result of the challenge was carried out earlier than the Amazon Bedrock Agent was typically obtainable. Nonetheless, Amazon Bedrock Brokers are actually the beneficial resolution for organizations wanting to make use of synthetic intelligence brokers of their operations. Amazon Bedrock Brokers retain reminiscence throughout interactions, offering a extra private and seamless person expertise. You profit from improved strategies and recall of earlier context when wanted, having fun with extra cohesive and environment friendly interactions with brokers. We share classes discovered from options that can assist you perceive easy methods to use AWS know-how to construct options to realize your targets.

Answer overview

Planview’s multi-agent structure consists of a number of generative AI parts that collaborate as a single system. At its core, the coordinator is accountable for routing inquiries to particular person brokers, gathering discovered data, and offering a complete response to the person. The orchestrator is managed by a central growth workforce, and brokers are managed by every software workforce.

The orchestrator consists of two primary elements referred to as routers and responders, that are powered by massive language fashions (LLM). The router makes use of synthetic intelligence to intelligently route person questions to varied software proxies with skilled options. Brokers will be divided into three primary sorts:

• assist agent – Present software help utilizing Retrieval Augmentation Technology (RAG)
• knowledge dealer – Dynamically entry and analyze buyer knowledge
• motion agent – Carry out actions inside the software on behalf of the person

After the agent processes the query and offers a solution, the responder, additionally powered by the LLM, synthesizes the knowledge discovered and formulates a coherent response to the person. This structure permits for seamless collaboration between a centralized coordinator and specialised brokers to supply correct and complete solutions to person questions. The diagram beneath illustrates the end-to-end workflow.

Technical overview

Planview makes use of key AWS providers to construct its multi-agent structure. The central Copilot service, powered by Amazon Elastic Kubernetes Service (Amazon EKS), coordinates exercise amongst numerous providers. Its tasks embrace:

• Handle person session chat historical past utilizing Amazon Relational Database Service (Amazon RDS)
• Coordinates site visitors between routers, software brokers, and responders
• Deal with logging, monitoring and gathering user-submitted suggestions

Routers and responders are AWS Lambda features that work together with Amazon Bedrock. The router considers person questions and chat historical past from the central Copilot service, whereas the responder considers person questions, chat historical past, and responses from every agent.

Software groups use Lambda features that work together with Amazon Bedrock to handle their brokers. To enhance visibility, evaluation, and monitoring, Planview makes use of a centralized tip repository service to retailer LLM suggestions.

Brokers can work together with purposes utilizing numerous strategies based mostly on use case and knowledge availability:

• Present software API – Brokers can talk with purposes by current API endpoints
• Amazon Athena or conventional SQL knowledge storage – Brokers can retrieve knowledge from Amazon Athena or different SQL-based knowledge shops to supply related data
• Amazon Neptune for graphic supplies – Brokers can entry graph knowledge saved in Amazon Neptune to help advanced dependency evaluation
• Amazon OpenSearch service for file RAG – Brokers can use Amazon OpenSearch Service to carry out RAG on paperwork

The determine beneath reveals the generative AI assistant structure on AWS.

Router and Responder Instance Suggestions

Router and responder parts work collectively to course of person queries and generate applicable responses. The next suggestions present pattern router and responder prompts. Extra just-in-time engineering is required to enhance the reliability of manufacturing implementation.

First, the obtainable instruments are described, together with their goal and the instance questions that every software can ask. Instance questions assist information pure language interactions between the coordinator and obtainable brokers (as proven within the software).

instruments=""'
<software>
<toolName>applicationHelp</toolName>
<toolDescription>
Use this software to reply software assist associated questions.
Instance questions:
How do I reset my password?
How do I add a brand new person?
How do I create a job?
</toolDescription>
</software>
<software>
<toolName>dataQuery</toolName>
<toolDescription>
Use this software to reply questions utilizing software knowledge.
Instance questions:
Which duties are assigned to me?
What number of duties are due subsequent week?
Which job is most in danger?
</toolDescription>
</software>

Subsequent, router suggestions define tips for brokers to reply on to person queries or request data by particular instruments earlier than formulating a response:

system_prompt_router = f'''
<position>
Your job is to resolve should you want extra data to totally reply the Consumer's 
questions.
You obtain your purpose by selecting both 'reply' or 'callTool'.
You will have entry to your chat historical past in <chatHistory></chatHistory> tags.
You even have a listing of accessible instruments to help you in <instruments></instruments> tags.
</position>
<chatHistory>
{chatHistory}
</chatHistory>
<instruments>
{instruments}
</instruments>
<guidelines>
- If the chat historical past accommodates ample data to reply the Consumer's questions, 
select the 'reply' motion.
- To assemble extra data earlier than responding, select the 'callTool' motion.
- You a lot solely select from the instruments within the <instruments></instruments> tags.
- If no software can help with the query, select the 'reply' motion.
- Place your chosen motion inside <motion></motion> tags.
- While you selected the 'callTool' motion, present the <toolName> and the <toolQuestion> you
want to ask.
- Your <toolQuestion> must be verbose and keep away from utilizing pronouns.
- Begin by offering your step-by-step considering in <considering></considering> tags.
- Then you'll give your reply in <reply></reply> tags.
- Your reply ought to comply with the format of one among these three examples:
When selecting the 'reply' motion, your reply ought to comply with the beneath instance EXACTLY:
<instance>
<reply>
<motion>reply</motion>
</reply>
</instance>
When selecting the 'callTool' motion for a single Device:
<instance>
<reply>
<motion>callTool</motion>
<instruments>
<software>
<toolName>applicationHelp</toolName>
<toolQuestion>How do I reset my password?</toolQuestion>
</software>
</instruments>
</reply>
</instance>
Executing the above, would produce the next outcome:
You can even name a number of Instruments utilizing this format:
<instance>
<reply>
<motion>callTool</motion>
<instruments>
<software>
<toolName>dataQuery</toolName>
<toolQuestion>What number of duties are assigned to me?</toolQuestion>
</software>
<software>
<toolName>applicationHelp</toolName>
<toolQuestion>How do I add a brand new job?</toolQuestion>
</software>
</instruments>
</reply>
</instance>
</guidelines>
'''

The next is an instance response from a Router element that launches the dataQuery software to retrieve and analyze every person’s job assignments:

<considering>
To find out who has probably the most duties assigned, I might want to question the applying knowledge. The "dataQuery" software appears most applicable for this query.
</considering>

<reply>
<motion>callTool</motion>
<instruments>
    <software>
        <toolName>dataQuery</toolName>
        <toolQuestion>Which person has probably the most duties at present assigned to them?   </toolQuestion>
    </software>
</instruments>
</reply>

The next is a pattern response from a responder element that makes use of the dataQuery software to acquire details about a user-assigned job. It experiences that the person has 5 duties assigned to him.

<considering>
Primarily based on the chat historical past, I beforehand referred to as the dataQuery software to ask "What number of duties are at present assigned to the person?". The software responded that the person has 5 duties assigned to them.
</considering>

<reply>
In accordance with the information I queried beforehand, you could have 5 duties assigned to you.
</reply>

Mannequin analysis and choice

Evaluating and monitoring the efficiency of generative AI fashions is crucial for any AI system. Planview’s multi-agent structure helps analysis on the particular person element degree, offering complete high quality management regardless of the system’s complexity. Planview evaluates elements at three ranges:

• trace – Consider the effectiveness and accuracy of LLM prompts
• synthetic intelligence agent – Consider full cue chains to keep up optimum job processing and response relevance
• synthetic intelligence system – Check user-facing interactions to confirm seamless integration of all elements

The diagram beneath illustrates the evaluation framework for prompts and scoring.

To conduct these evaluations, Planview makes use of a fastidiously designed set of check questions that cowl typical person queries and edge circumstances. These evaluations are performed in the course of the growth part and proceed in manufacturing to trace response high quality over time. At the moment, human evaluators play a vital position in scoring responses. To help evaluation, Planview developed an inside evaluation software to retailer a financial institution of questions and monitor solutions over time.

To judge every element and decide the Amazon Bedrock mannequin greatest fitted to a given job, Planview established the next precedence analysis standards:

• response high quality – Guarantee system responses are correct, related and helpful
• response time – Decrease delays between person queries and system responses
• scale – Make sure the system can scale to hundreds of concurrent customers
• response price – Optimize working prices, together with AWS providers and generative AI fashions, to keep up financial viability

Primarily based on these standards and present use circumstances, Planview chosen Anthropic’s Claude 3 Sonnet on Amazon Bedrock for the router and responder parts.

outcomes and influence

Over the previous yr, Planview Copilot has considerably improved its efficiency by implementing a multi-agent structure, growing a strong analysis framework, and adopting the most recent FM from Amazon Bedrock. Planview achieved the next outcomes between the primary technology of Planview Copilot developed in mid-2023 and the most recent model:

• accuracy – Human evaluation accuracy has elevated from 50% reply acceptance fee to now over 95%
• response time – Common response time diminished from over 1 minute to twenty seconds
• load check – The AI ​​assistant has efficiently handed the load check, submitting 1,000 questions concurrently with no vital influence on response time or high quality
• Price efficient – The fee per buyer interplay has been diminished to one-tenth of the preliminary price
• time to market – Improvement and deployment time for brand spanking new brokers has been diminished from months to weeks

in conclusion

On this article, we discover how Planview solves advanced work administration processes by growing a generative AI assistant utilizing the next methods:

• Modular growth – Planview makes use of a centralized coordinator to construct a multi-agent structure. The answer allows environment friendly job processing and system scalability, whereas permitting totally different product groups to rapidly develop and deploy new AI abilities by devoted brokers.
• Evaluation framework – Planview has carried out a strong analysis course of at a number of ranges, which is crucial to sustaining and enhancing efficiency.
• Amazon Bedrock Integration – Planview makes use of Amazon Bedrock to speed up innovation by a broad choice of fashions and entry to a wide range of FMs, giving the flexibleness to decide on fashions based mostly on particular mission necessities.

Planview is migrating to Amazon Bedrock Brokers, which allows the combination of good autonomous brokers into its software ecosystem. Amazon Bedrock Brokers automate processes by coordinating interactions between underlying fashions, knowledge sources, purposes, and person conversations.

Subsequent, you possibly can discover the Planview AI Assistant options constructed on Amazon Bedrock, and keep knowledgeable about new Amazon Bedrock options and variations to advance your AI journey on AWS.

In regards to the writer

Sunil Ramachandra is a senior options architect who helps high-growth unbiased software program distributors (ISVs) innovate and speed up on AWS. He works with clients to construct extremely scalable and resilient cloud architectures. When not working with purchasers, Sunil enjoys spending time together with his household, operating, meditating, and watching motion pictures on Prime Video.

Benedict Augustine He’s a thought chief within the discipline of generative AI and machine studying and serves as a senior knowledgeable at AWS. As Vice President of Machine Studying, Benedict has constructed seven AI-first SaaS merchandise over the previous decade that are actually utilized by Fortune 100 firms, leading to vital enterprise influence. His work earned him 5 patents.

Lee Ray Winkle is the Chief Knowledge Scientist at Planview with 20 years of expertise integrating AI and ML into enterprise software program. He holds superior levels from Carnegie Mellon College and Columbia College. Lee leads the analysis and growth of synthetic intelligence capabilities inside Planview Copilot. Outdoors of labor, he enjoys boating on Girl Hen Lake in Austin.