With the rise of highly effective base fashions (FMs) powered by companies like Amazon Bedrock and Amazon SageMaker JumpStart, enterprises need granular management over which customers and teams can entry and use these fashions. That is vital for compliance, safety and governance.
Launched in 2021, Amazon SageMaker Canvas is a visible point-and-click service that permits enterprise analysts and citizen knowledge scientists to make use of ready-to-use machine studying (ML) fashions and construct customized ML fashions to generate correct predictions with out writing any code. SageMaker Canvas gives a code-free interface for out-of-the-box use of assorted FMs from each companies, in addition to retrieval-augmented technology (RAG) workflows utilizing Amazon Kendra as a data base to customise mannequin responses or use tagged knowledge Set for fine-tuning. This simplifies entry to generative synthetic intelligence (AI) capabilities for enterprise analysts and knowledge scientists with out requiring technical data or writing code, rising productiveness.
On this article, we look at methods for utilizing AWS Identification and Entry Administration (IAM) insurance policies to handle entry to Amazon Bedrock and SageMaker JumpStart fashions from SageMaker Canvas. You may learn to set up fine-grained permissions to regulate invocation of out-of-the-box Amazon Bedrock fashions and stop the SageMaker endpoint from being provisioned with a specified SageMaker JumpStart mannequin. We offer code examples tailor-made for widespread company governance situations. Lastly, you will learn to lock down entry to generative AI capabilities primarily based in your organizational wants, sustaining protected and compliant use of cutting-edge AI in a code-free SageMaker Canvas setting.
This text covers an more and more necessary matter as extra highly effective AI fashions change into out there, making it a priceless useful resource for machine studying operators, safety groups, and AI managers within the enterprise.
Resolution overview
The diagram under exhibits the structure of the answer.
SageMaker Canvas’ structure permits enterprise analysts and knowledge scientists to work together with ML fashions with out writing any code. Nonetheless, managing entry to those fashions is vital to sustaining safety and compliance. When customers work together with SageMaker Canvas, the actions they carry out (similar to calling a mannequin or establishing an endpoint) are run by SageMaker service roles. SageMaker consumer profiles can inherit the default roles of the SageMaker area, or they will have user-specific roles.
Managing entry to generative AI capabilities by customizing the coverage connected to this function lets you management which actions are allowed or denied. As a part of this text, we focus on which IAM insurance policies this function makes use of to regulate operations inside SageMaker Canvas, similar to calling fashions or establishing endpoints primarily based on enterprise group necessities. We analyzed two patterns for Amazon Bedrock fashions and SageMaker JumpStart fashions: limiting entry to all fashions from the service or limiting entry to particular fashions.
Managing Amazon Bedrock’s entry to SageMaker Canvas
To make use of the Amazon Bedrock mannequin, SageMaker Canvas calls the next Amazon Bedrock API:
- Cornerstone: InvokeModel – Synchronous name mannequin
- Cornerstone: InvokeModelWithResponseStream – Synchronous name mannequin, the response is transmitted by way of the socket, as proven within the determine under
As well as, SageMaker Canvas can name the bedrock:FineTune API to fine-tune massive language fashions (LLM) utilizing Amazon Bedrock. As of this writing, SageMaker Canvas solely permits fine-tuning of the Amazon Titan mannequin.
To make use of a selected LLM from Amazon Bedrock, SageMaker Canvas makes use of the mannequin ID of the chosen LLM as a part of the API name. As of this writing, SageMaker Canvas helps the next fashions in Amazon Bedrock (grouped by mannequin supplier):
- Synthetic Intelligence 21
- Jurassic 2:
j2-mid-v1
- Jurassic 2 Extremely:
j2-ultra-v1
- Jurassic 2:
- Amazon
- Titan:
titan-text-premier-v1:*
- Titan Massive:
titan-text-lite-v1
- Titan Categorical:
titan-text-express-v1
- Titan:
- Anthropic choice
- Claude 2:
claude-v2
- Claude Immediate:
claude-instant-v1
- Claude 2:
- coherent
- Command textual content:
command-text-*
- Command gentle:
command-light-text-*
- Command textual content:
- Yuan
- Alpaca 2 13B:
llama2-13b-chat-v1
- Camel 2 70B:
llama2-70b-chat-v1
- Alpaca 2 13B:
For a whole record of Amazon Bedrock mannequin IDs, see Amazon Bedrock Mannequin IDs.
Limit entry to all Amazon Bedrock fashions
To limit entry to all Amazon Bedrock fashions, you’ll be able to modify your SageMaker function to explicitly deny these APIs. This ensures that no consumer can name any Amazon Bedrock mannequin by way of SageMaker Canvas.
Right here is an instance IAM coverage to perform this:
{
"Model": "2012-10-17",
"Assertion": [
{
"Effect": "Deny",
"Action": [
"bedrock:InvokeModel",
"bedrock:InvokeModelWithResponseStream"
],
"Useful resource": "*"
}
]
}
This technique makes use of the next parameters:
"Impact": "Deny"
Specify that the next operations are denied"Motion": ["bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream"]
Specify rejected Amazon Bedrock API"Useful resource": "*"
Rejection applies to all Amazon Bedrock fashions
Limit entry to particular Amazon Bedrock fashions
You may lengthen the IAM coverage above to limit entry to particular Amazon Bedrock fashions by specifying the mannequin ID within the useful resource portion of the coverage. This fashion, customers can solely name allowed fashions.
The next is an instance of extending an IAM coverage:
{
"Model": "2012-10-17",
"Assertion": [
{
"Effect": "Deny",
"Action": [
"bedrock:InvokeModel",
"bedrock:InvokeModelWithResponseStream"
],
"Useful resource": [
"arn:aws:bedrock:<region-or-*>::foundation-model/<model-id-1>",
"arn:aws:bedrock:<region-or-*>::foundation-model/<model-id-2>"
]
}
]
}
On this coverage, the Useful resource array lists the particular Amazon Bedrock fashions that have been rejected. Present the AWS Area, Account, and Mannequin ID applicable to your setting.
Managing SageMaker JumpStart entry to SageMaker Canvas
To ensure that SageMaker Canvas to make use of LLM from SageMaker JumpStart, it should do the next:
- Choose LLM from the SageMaker Canvas or JumpStart mannequin ID record (hyperlink under).
- Construct endpoint configuration and deploy LLM on dwell endpoints.
- Name the endpoint to generate predictions.
The diagram under illustrates this workflow.
For an inventory of obtainable JumpStart mannequin IDs, see the JumpStart Obtainable Fashions desk. As of this writing, SageMaker Canvas helps the next mannequin IDs:
huggingface-textgeneration1-mpt-7b-*
huggingface-llm-mistral-*
meta-textgeneration-llama-2-*
huggingface-llm-falcon-*
huggingface-textgeneration-dolly-v2-*
huggingface-text2text-flan-t5-*
To establish the proper mannequin from SageMaker JumpStart, SageMaker Canvas aws:RequestTag/sagemaker-sdk:jumpstart-model-id
As a part of the endpoint configuration. To study extra about different strategies for utilizing IAM permissions to limit entry to SageMaker JumpStart fashions, see Managing Amazon SageMaker JumpStart base mannequin entry utilizing a personal middle.
Configuring permissions for deployment endpoints by way of the UI
On the SageMaker Area Settings web page on the SageMaker web page of the AWS Administration Console, you’ll be able to arrange SageMaker Canvas to have the ability to deploy SageMaker endpoint nodes. This selection additionally helps the deployment of real-time endpoints for traditional machine studying fashions, similar to time collection forecasting or classification. To allow mannequin deployment, full the next steps:
- On the Amazon SageMaker console, navigate to your area.
- superior Area title particulars web page, choose Utility configuration
- exist canvas half, choose edit.
- Open Allow direct deployment of Canvas fashions exist Machine studying operation configuration
Limit entry to all SageMaker JumpStart fashions
To limit entry to all SageMaker JumpStart fashions, configure the SageMaker function to Block CreateEndpointConfig
and CreateEndpoint
API on any SageMaker JumpStart mannequin ID. This prevents endpoints from being constructed utilizing these fashions. Please have a look at the next code:
{
"Model": "2012-10-17",
"Assertion": [
{
"Effect": "Deny",
"Action": [
"sagemaker:CreateEndpointConfig",
"sagemaker:CreateEndpoint"
],
"Useful resource": "*",
"Situation": {
"Null": {
"aws:RequestTag/sagemaker-sdk:jumpstart-model-id":”*”
}
}
}
]
}
This technique makes use of the next parameters:
"Impact": "Deny"
Specify that the next operations are denied"Motion": ["sagemaker:CreateEndpointConfig", "sagemaker:CreateEndpoint"]
Specify rejected SageMaker APIs- this
"Null"
Conditional operators in AWS IAM insurance policies are used to examine whether or not the important thing exists. It doesn’t examine the worth of the important thing, solely its presence or absence "aws:RequestTag/sagemaker-sdk:jumpstart-model-id":”*”
Rejection applies to all SageMaker JumpStart fashions
Limit entry and deployment of particular SageMaker JumpStart fashions
Just like Amazon Bedrock fashions, you’ll be able to limit entry to particular SageMaker JumpStart fashions by specifying the mannequin ID in an IAM coverage. To attain this, directors want to limit customers from utilizing unauthorized fashions to create endpoints. For instance, to disclaim entry to the Hugging Face FLAN T5 mannequin and MPT mannequin, use the next code:
{
"Model": "2012-10-17",
"Assertion": [
{
"Effect": "Deny",
"Action": [
"sagemaker:CreateEndpointConfig",
"sagemaker:CreateEndpoint"
],
"Useful resource": "*",
"Situation": {
"StringLike": {
"aws:RequestTag/sagemaker-sdk:jumpstart-model-id": [
"huggingface-textgeneration1-mpt-7b-*",
"huggingface-text2text-flan-t5-*"
]
}
}
}
]
}
On this coverage, "StringLike"
Situations permit sample matching, enabling insurance policies to be utilized to a number of mannequin IDs with comparable prefixes.
clear up
To keep away from future workspace occasion costs, log off of SageMaker Canvas if you find yourself executed utilizing the appliance. Alternatively, you’ll be able to set SageMaker Canvas to routinely shut when idle.
in conclusion
On this article, we exhibit how SageMaker Canvas calls LLM powered by Amazon Bedrock and SageMaker JumpStart, and the way enterprises can handle entry to those fashions, whether or not you need to limit entry to a selected mannequin or any mannequin from both service go to. You may mix the IAM insurance policies proven on this article into the identical IAM function to offer full management.
By following these pointers, enterprises can be sure that their use of generative AI fashions is each protected and in keeping with organizational insurance policies. This method not solely protects delicate knowledge, but additionally permits enterprise analysts and knowledge scientists to appreciate the total potential of synthetic intelligence in a managed setting.
Now that your setting is configured in response to enterprise requirements, we suggest that you simply learn the next articles to study what SageMaker Canvas might help you do with generative AI:
Concerning the creator
David Garlitelli is a senior knowledgeable answer architect for GenAI/ML. An Italian native dwelling in Brussels, he works carefully with clients around the globe on generative AI workloads and low-code no-code ML applied sciences. He has been creating since he was a baby and began writing code on the age of seven.
Lijan Kunyil is a Senior Technical Account Supervisor at AWS. Lijan enjoys serving to AWS enterprise clients construct extremely dependable, cost-effective methods and obtain operational excellence. Lijan has over 25 years of expertise creating options for monetary and consulting companies.
Saptash Banerjee Serves as a Senior Companion Options Architect at AWS, working carefully with AWS companions to design and construct mission-critical options. Saptarshi makes a speciality of generative AI, AI/ML, serverless structure, and cloud-based options to enhance efficiency, innovation, scalability, and value effectivity for AWS companions throughout the cloud ecosystem.