Decoding Amazon Q for Business
A fascinating AWS Reinvent 2023 announcement in the Generative AI space was Amazon Q. This post is an early attempt to decipher the motivation and methodology of Amazon Q in democratizing Generative AI.
Customizing Generative AI applications to your enterprise data requires you to either Fine Tuning Large Language model or leverage Retrieval Augmented Generation [RAG} pattern. RAGs application extends from building
In one of the previous posts, we discussed the difference between building a RAG POC vs. building a production grade RAG application. A production grade RAG application demands the following capabilities
- Connectors — Ability to connect to your enterprise data store [structured from transactional and analytical databases, unstructured from Object stores, Code bases and SaaS platforms…] to ingest data in to the knowledge store. Each Systems of Record demands a connectivity pattern and configurations.
- Data Processing: Data in the SORs can in any shape or form. For eg. Your S3 bucket might have data in PDF, scanned images, docs, ppts etc. You will have define a set of data processing techniques to extract, process and ready the data for indexing.
- Embedding Model — to create dense vector embeddings / enrich features for the data ingested from the SORs
- Knowledge Base — to index the vector embeddings, the associated text, meta data in performant / cost optimized way for search and retrieval.
- Scheduler — to sync updates from the SORs to the Knowledge base supporting both schedule and event based.
- Retriever — to retrieve the relevant context from the knowledge base for the query / prompt from the user, rank the responses based on business requirements
- Generation Model — to take the context and the prompt to generate and format the response for the user
- Guardrails — to ensure the query / prompt, the context retrieved from the knowledge base, generated response from the Generation model is accurate, responsible and ethical free is hallucinations, bias….
- Orchestrator — to automate the end-to-end workflow — the prompts from the user, retrieve the relevant context from the knowledge base and format the response to the user
- User Experience — A conversational Chat Interface with rich features from displaying chat history, to capturing feedback on responses from the user
- Context Management [Memory] — Session management, context within the session, to enrich conversations
- User Management: to address who can access the application
Remember… all I’m offering is the truth. Nothing more….
In addition to the functional capabilities of RAG, you will have to consider the non-functional requirements. Iterating Werner Vogels quote from reinvent, developing an application is 25% effort, 75% effort is operating the system at scale
They won’t fear it until they understand it, and they won’t understand it until they’ve used it
You need a LOT of moving parts to build your own enterprise grade chatbot. Standing up a RAG solution and operating the same requires considerable heavy lifting from people, processes, technologies driving longer time to market your solution. Now imagine scaling this to 100s of use cases becomes daunting to say the least.
Amazon Q is a managed Generative AI Service that is designed to leverage your Enterprise Content for Gen AI use cases.
Amazon Q compresses all of these complexities into one simple experience. Amazon Q does the undifferentiated heavy lifting you can now focus on building your Gen AI application. Amazon Q differentiator lies in providing built in capabilities and an ability to configure based on your business requirements.
Using Amazon Q, you can build your production grade RAG based Generative AI application in few hours
- Built in connectors: Amazon Q for business has 25+ connectors out of the box including Adobe (AEM Connectors), SalesForce and a Custom connector capability. Full list of Amazon Q connectors are here -https://docs.aws.amazon.com/amazonq/latest/business-use-dg/connectors-list.html. In the case of non availability of out of the box connector, the design pattern is to leverage Amazon AppFlow as a intermediary to pull data to Amazon S3. Full list of Amazon AppFlow connectors are here -https://aws.amazon.com/appflow/integrations/
- Built in pipelines: Amazon Q provides an inbuilt pipeline for indexing data in to knowledge base. The indexing pipeline is built to index your ACLs [Access control lists] there by porting the access management you have set for the datasets to the index. If you chose to add pre processing logic for your indexing pipeline, you can chose do so using a pre-processor lambda.
- Built in Knowledge base: There is no need to stand up separate knowledge base. Amazon Q has a built in knowledge base. If you chose to use an existing knowledge base, you can configure Kendra as the knowledge base.
- Models: Amazon Q leverages Amazon Bedrock for the text generation models. At this time of writing, the model is abstracted from the end user.
- Plugins: Provide capability to automate tasks on target systems [Like a RPA capability]. For e.g. Summarize the ticket information and creating a ticket in Jira. Amazon Q support 4 out of the box plugins. List of plug in support is listed here -https://docs.aws.amazon.com/amazonq/latest/business-use-dg/plugins.html
- Guardrails: Amazon Q support Global and topic level guardrails to address PII detection, abuse, sensitive information in prompts. hallucinations. Details on guard rails are available here — https://docs.aws.amazon.com/amazonq/latest/business-use-dg/guardrails-global-controls.html
- User Management: With Amazon Q you can mange users and their access to application.
There’s a difference between knowing the path and walking the path
The below video is a quick overview of how to set up an application in Amazon Q for business in few minutes.
Once the Application is set up in Amazon Q, the below video talks shows the user experience of Amazon Q. In addition it touches upon some of the additional configurations you can do on Amazon Q
Amazon Q is everywhere. It is all around us. Even now in this very room —
With Amazon Q as a framework, AWS built Amazon Q for builders. Amazon Q for builders is built on wealth of AWS expertise from product documentation to FAQs to Well Architected principles. Q for builders is designed to help with end to end developer experience
- Getting to know about AWS Services: Before diving in to building with AWS Services, you are looking for information on AWS Services and how to work with. You can ask questions using Amazon Q for AWS console
- Advising while configuring the services. For e.g. when you provision your EC2 instances, you will see “Get Advice” option to get advise on the sizing of the EC2 instances from Amazon Q
- Trouble shooting errors when using AWS services. For e.g. when you encounter errors with network reachability errors or lambda executions, you can launch Amazon Q to get recommendations on trouble shooting the errors
- Assist in coding: Once AWS services are configured, you start to develop your application. Amazon Q for Code whisperer can configured in IDE’s of your choice and helps as a coding assistant to provide code recommendations and chat with improving the quality of your code.
- Assist in SQL Development: If you chose to build your SQL code using Amazon Redshift, Amazon Q for Redshift
- Amazon Q for Code Catalyst: Amazon Q feature development capability in Amazon CodeCatalyst is a generative AI assistant to which you can assign issues. Once an issue is assigned to Amazon Q, it will analyze the issue based on its title and its description and review the code in the specified repository. If it can create an approach, it will create a draft solution for users to evaluate in a pull request
Amazon Q is transforming the way you do business or build on AWS. To get started with Amazon Q for Business, go to https://aws.amazon.com/q/business-expert/
