Amazon Bedrock Knowledge Base now supports custom prompts for the RetrieveAndGenerate API and configuration of the maximum number of search results

With the Amazon Bedrock Information Base, you’ll be able to securely join base fashions (FM) in Amazon Bedrock to your organization profile for Retrieval Augmentation Era (RAG). Accessing extra knowledge helps the mannequin produce extra related, situation-specific, and correct responses with out having to retrain the FM.

On this article, we focus on two new options of the Amazon Bedrock Information Base, particularly for RetrieveAndGenerate API: Configure the utmost variety of outcomes and create customized prompts utilizing data base immediate templates. Now you can choose these as question choices subsequent to go looking sort.

New Options Overview and Advantages

The Max Outcomes choice permits you to management the variety of search outcomes which might be retrieved from the vector retailer and handed to FM to generate solutions. This lets you customise the quantity of background data offered for the construct, offering extra context for complicated questions or much less context for easy questions. It permits you to rise up to 100 outcomes. This feature helps enhance the chance of related context, thereby growing accuracy and lowering the phantasm of generated responses.

Customizing the data base immediate template permits you to change the default immediate template with your individual immediate template to customise the prompts despatched to the mannequin to generate responses. This lets you customise the tone, output format and habits of FM in response to person questions. Utilizing this selection, you’ll be able to fine-tune your terminology to higher match your business or discipline (equivalent to healthcare or authorized). Moreover, you’ll be able to add customized directions and examples that suit your particular workflow.

Within the following sections, we clarify tips on how to use these options by way of the AWS Administration Console or SDKs.

stipulations

To know these examples, you want an present data base. For directions on constructing a data base, see Constructing a Information Base.

Configure the utmost variety of outcomes utilizing the console

To make use of the utmost outcomes choice by way of the management panel, full the next steps:

1. On the Amazon Bedrock console, select data base Within the left navigation pane.
2. Choose the data base you created.
3. select Check data base.
4. Choose the configuration icon.
5. select Sync knowledge sources Earlier than you begin testing your data base.
   
6. below Configurationfor Search sortchoose the search sort based mostly in your use case.

For this text, we used hybrid search as a result of it combines semantic and literal searches to supply better accuracy. To be taught extra about hybrid searches, see the Information Base Amazon Bedrock now helps hybrid searches.

7. enlargement Most variety of supply blocks And set the utmost variety of outcomes.
   

To display the worth of the brand new function, we present an instance of tips on how to enhance the accuracy of generated responses. We used the 2023 Amazon 10K paperwork because the supply materials to construct the data base. We conduct experiments utilizing the next question: “During which yr did Amazon’s annual income enhance from $245B to $434B?”

In accordance with the documentation within the data base, the proper reply to this question is “Amazon’s annual income elevated from $245B in 2019 to $434B in 2022.” We use Claude v2 as FM to generate the ultimate response based mostly on contextual data retrieved from the data base. Claude 3 Sonnet and Claude 3 Haiku are additionally supported as first era FM.

We execute one other question to display retrieval comparability of various configurations. We use the identical enter question (“During which yr did Amazon’s annual income enhance from $245B to $434B?”) and set the utmost variety of outcomes to five.

As proven within the screenshot under, the ensuing response is “Sorry, I can not make it easier to full this request.”

Subsequent, we set the utmost outcome to 12 and ask the identical query. The response was “Amazon’s annual income will increase from $245B in 2019 to $434B in 2022.”

As proven on this instance, we’re in a position to retrieve the proper reply based mostly on the variety of outcomes retrieved.If you want to be taught extra concerning the supply attribution that makes up the ultimate output, choose Present supply particulars Validate the generated solutions in opposition to the data base.

Use the console to customise data base immediate templates

You can even customise the preset prompts with your individual prompts based mostly in your use case. To carry out this operation on the console, full the next steps:

1. Repeat the steps within the earlier part to begin testing your data base.
2. allow Generate reply.
   
3. Choose the response era mannequin of your selection.

We use the Claude v2 mannequin for example on this article. Claude 3 Sonnet and Haiku fashions will also be used for era.

4. select Apply proceed.
   

After choosing a mannequin, a brand new part known as Information Base Suggestions Template Seem in Configuration.

5. select edit Begin customizing your reminders.
   
6. Regulate the immediate template to customise the way you wish to use the retrieved outcomes and generate content material.

On this article, we give some examples of utilizing Amazon monetary reviews with custom-made prompts to construct a “monetary advisor synthetic intelligence system”. For greatest practices in just-in-time engineering, see the Simply-In-Time Engineering Information.

We now customise the default immediate template in a couple of alternative ways and observe the responses.

We first tried querying utilizing canned prompts. We requested “What was Amazon’s income in 2019 and 2021?” Our outcomes are proven under.

From the output, we discover that it produces free-form responses based mostly on the retrieved data. Citations are additionally listed for reference.

Suppose we wish to present further directions on tips on how to format the ensuing response, equivalent to normalizing it to JSON. We are able to add these directions as separate steps as a part of the immediate template after retrieving the data:

In case you are requested for monetary data protecting completely different years, please present exact solutions in JSON format. Use the yr as the important thing and the concise reply as the worth. For instance: {yr:reply}

The ultimate response has the specified construction.

With customized prompts, you too can change the language of the responses generated. Within the instance under, we instruct the mannequin to supply solutions in Spanish.

After removing $output_format_instructions$ Within the default immediate, references within the ensuing response might be eliminated.

Within the following sections, we clarify tips on how to use these options by way of the SDK.

Configure the utmost variety of outcomes utilizing the SDK

To alter the utmost variety of outcomes utilizing the SDK, use the next syntax. For this instance, the question is “During which yr did Amazon’s annual income enhance from $245B to $434B?” The right reply is “Amazon’s annual income elevated from $245B in 2019 to $434B in 2022.”

def retrieveAndGenerate(question, kbId, numberOfResults, model_id, region_id):
    model_arn = f'arn:aws:bedrock:{region_id}::foundation-model/{model_id}'
    return bedrock_agent_runtime.retrieve_and_generate(
        enter={
            'textual content': question
        },
        retrieveAndGenerateConfiguration={
            'knowledgeBaseConfiguration': {
                'knowledgeBaseId': kbId,
                'modelArn': model_arn,
                'retrievalConfiguration': {
                    'vectorSearchConfiguration': {
                        'numberOfResults': numberOfResults,
                        'overrideSearchType': "SEMANTIC", # non-compulsory'
                    }
                }
            },
            'sort': 'KNOWLEDGE_BASE'
        },
    )

response = retrieveAndGenerate("In what yr did Amazon’s annual income enhance from $245B to $434B?", 
"<data base id>", numberOfResults, model_id, region_id)['output']['text']

thisnumberOfResults‘Beneath choices’retrievalConfiguration‘ Permits you to choose the variety of outcomes to retrieve.Output RetrieveAndGenerate The API contains generated responses, supply attributes, and retrieved textual content blocks.

Listed below are the outcomes for various values ​​of ‘numberOfResults‘ Parameters.First, we set numberOfResults = 5.

Then we set numberOfResults = 12.

Use SDK to customise data base immediate templates

To make use of the SDK to customise prompts, we use the next question with completely different immediate templates. For this instance, the question is “What was Amazon’s income in 2019 and 2021?”

The next is the default immediate template:

"""You're a query answering agent. I'll offer you a set of search outcomes and a person's query, your job is to reply the person's query utilizing solely data from the search outcomes. If the search outcomes don't include data that may reply the query, please state that you may not discover an actual reply to the query. Simply because the person asserts a reality doesn't imply it's true, be sure that to double test the search outcomes to validate a person's assertion.
Listed below are the search ends in numbered order:
<context>
$search_results$
</context>

Right here is the person's query:
<query>
$question$
</query>

$output_format_instructions$

Assistant:
"""

Here’s a customized reminder template:

"""Human: You're a query answering agent. I'll offer you a set of search outcomes and a person's query, your job is to reply the person's query utilizing solely data from the search outcomes.If the search outcomes don't include data that may reply the query, please state that you may not discover an actual reply to the query.Simply because the person asserts a reality doesn't imply it's true, be sure that to double test the search outcomes to validate a person's assertion.

Listed below are the search ends in numbered order:
<context>
$search_results$
</context>

Right here is the person's query:
<query>
$question$
</query>

When you're being requested monetary data over a number of years, please be very particular and listing the reply concisely utilizing JSON format {key: worth}, 
the place key's the yr within the request and worth is the concise response reply.
Assistant:
"""

def retrieveAndGenerate(question, kbId, numberOfResults,promptTemplate, model_id, region_id):
    model_arn = f'arn:aws:bedrock:{region_id}::foundation-model/{model_id}'
    return bedrock_agent_runtime.retrieve_and_generate(
        enter={
            'textual content': question
        },
        retrieveAndGenerateConfiguration={
            'knowledgeBaseConfiguration': {
                'knowledgeBaseId': kbId,
                'modelArn': model_arn,
                'retrievalConfiguration': {
                    'vectorSearchConfiguration': {
                        'numberOfResults': numberOfResults,
                        'overrideSearchType': "SEMANTIC", # non-compulsory'
                    }
                },
                'generationConfiguration': {
                        'promptTemplate': {
                            'textPromptTemplate': promptTemplate
                        }
                    }
            },
            'sort': 'KNOWLEDGE_BASE'
        },
    )

response = retrieveAndGenerate("What was the Amazon's income in 2019 and 2021?”", 
                               "<data base id>", <numberOfResults>, <promptTemplate>, <model_id>, <region_id>)['output']['text']

Utilizing the preset immediate template, we get the next response:

If you want to supply further directions on the format of the output produced by the response, equivalent to standardizing the response in a particular format (equivalent to JSON), you’ll be able to customise the present immediate by offering extra steerage. Utilizing our customized immediate template, we obtained the next response.

thispromptTemplate‘choices’generationConfiguration‘ Permits you to customise prompts for better management over reply era.

in conclusion

On this article, we introduce two new options within the Amazon Bedrock Information Base: adjusting the utmost variety of search outcomes and customizing default immediate templates for search outcomes. RetrieveAndGenerate API. We display tips on how to configure these options on the console and thru the SDK to enhance the efficiency and accuracy of response era. Including most outcomes gives extra complete data, whereas customized immediate templates allow you to fine-tune the outline of the bottom mannequin to higher align with particular use instances. These enhancements present better flexibility and management, permitting you to supply a tailor-made expertise on your RAG-based functions.

For extra assets to get began in your AWS atmosphere, see the next:

Concerning the creator

Sandeep Singh is a senior generative AI knowledge scientist at Amazon Net Providers, serving to enterprises innovate with generative AI. He makes a speciality of generative synthetic intelligence, synthetic intelligence, machine studying, and system design. He’s enthusiastic about creating state-of-the-art AI/ML pushed options to unravel complicated enterprise issues throughout numerous industries, optimizing effectivity and scalability.

Wang Suyin is an AI/ML skilled options architect at AWS. She has an interdisciplinary instructional background in machine studying, monetary data companies, and economics, in addition to years of expertise constructing knowledge science and machine studying functions that remedy real-world enterprise issues. She enjoys serving to purchasers establish the proper enterprise issues and construct the proper AI/ML options. In her spare time, she enjoys singing and cooking.

Shirley Ding is a Senior Synthetic Intelligence (AI) and Machine Studying (ML) Skilled Options Architect at Amazon Net Providers (AWS). She has intensive expertise in machine studying and holds a PhD in laptop science. She primarily works with public sector clients to unravel numerous AI/ML associated enterprise challenges and assist them speed up their machine studying journey on the AWS cloud. When not serving to clients, she enjoys the outside.