Andrew T.

Commented on Troubleshooting LLM Setup Issues in AWS Bedrock fo...·Posted inArize AX Support

Sounds good! I with The Hartford working with Samantha White on this. I am free today, tmrw, and Wednesday (up until noon) then I am OOO until April 21 😊

Commented on Troubleshooting LLM Setup Issues in AWS Bedrock fo...·Posted inArize AX Support

Andrew T.

Update: After some debugging it seems once I toggle the max_tokens the malformed input due to missing max_tokens key is resolved- guessing that once I do the toggle maybe something in the backend is triggered 😎. Curious if this is a bug 🐛. Next Problem: Seeing the same error with my eval that is using this prompt. Error: The required keys Prompt and max_tokens_to_sample are not found. It also says some extraneous keys are being provided like inputText and textGenerationConfig

Posted in Arize AX Support·

Andrew T.

Troubleshooting LLM Setup Issues in AWS Bedrock for Evaluation

Hello all! 👋 Context: Setting up LLM as a Judge through AWS Bedrock, with the end goal of running this as an online evaluation. Using Haiku 3.5. Problem: Testing out the prompt in the prompt playground prior to using this for Online Eval. Seeing a malformed input request regarding needing the key max_tokens, but this is already configured by default. Bedrock logs confirm the validation error on my side- looking for some help here. Doing this in the UI Things Tried: I can confirm in the UI that the input variables exist when trying this out; I've gone ahead and removed everything in the prompt so that it is just a string with no variables; still seeing the malformed input request.

2Comments