I'm using llamatrace, what do I use in this code block to replace px.Client() since when I use it goes back to my local host: # setup Arize Phoenix for logging/observability import llama_index.core import os PHOENIX_API_KEY = os.getenv("PHOENIX_API_KEY") os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = f"api_key={PHOENIX_API_KEY}" llama_index.core.set_global_handler( "arize_phoenix", endpoint="https://llamatrace.com/v1/traces" )
from phoenix.evals import (
HallucinationEvaluator,
OpenAIModel,
QAEvaluator,
RelevanceEvaluator,
run_evals,
)
from phoenix.session.evaluation import get_qa_with_reference, get_retrieved_documents
from phoenix.trace import DocumentEvaluations, SpanEvaluations
from tqdm import tqdm
import phoenix as px
# Define the evaluation model
eval_model = OpenAIModel(
azure_endpoint=AZURE_OPENAI_ENDPOINT,
azure_deployment="gpt-4o", # I'm using gpt-4o for evaluation
model=AZURE_OPENAI_CHAT_COMPLETION_DEPLOYED_MODEL_NAME,
api_key=AZURE_OPENAI_API_KEY,
api_version="2024-02-01"
)
# Define evaluators
hallucination_evaluator = HallucinationEvaluator(eval_model)
qa_correctness_evaluator = QAEvaluator(eval_model)
relevance_evaluator = RelevanceEvaluator(eval_model)
# List of project names
projects = ["Baseline", "Contextual"]
# Create DataFrames for queries and retrieved documents for each project
queries_dfs = {}
retrieved_documents_dfs = {}
for project in projects:
queries_dfs[project] = get_qa_with_reference(px.Client(), project_name=project)
retrieved_documents_dfs[project] = get_retrieved_documents(px.Client(), project_name=project)
# Loop through each project and perform evaluations
for project in projects:
queries_df = queries_dfs[project]
retrieved_documents_df = retrieved_documents_dfs[project]
# Run evaluations
hallucination_eval_df, qa_correctness_eval_df = run_evals(
dataframe=queries_df,
evaluators=[hallucination_evaluator, qa_correctness_evaluator],
provide_explanation=True,
)
relevance_eval_df = run_evals(
dataframe=retrieved_documents_df,
evaluators=[relevance_evaluator],
provide_explanation=True,
)[0]
# Log evaluations
px.Client().log_evaluations(
SpanEvaluations(eval_name=f"Hallucination_{project}", dataframe=hallucination_eval_df),
SpanEvaluations(eval_name=f"QA Correctness_{project}", dataframe=qa_correctness_eval_df),
DocumentEvaluations(eval_name=f"Relevance_{project}", dataframe=relevance_eval_df),
)error:onnectError: [WinError 10061] No connection could be made because the target machine actively refused it
To connect to a remote Phoenix instance instead of the local one, you need to specify the correct endpoint URL when initializing the px.Client. Replace the px.Client() calls with px.Client(endpoint="[https://llamatrace.com/v1/traces](https://llamatrace.com/v1/traces)") in your code. Here's the updated code block: ```from phoenix.evals import (
HallucinationEvaluator,
OpenAIModel,
QAEvaluator,
RelevanceEvaluator,
run_evals,
)
from phoenix.session.evaluation import get_qa_with_reference, get_retrieved_documents
from phoenix.trace import DocumentEvaluations, SpanEvaluations
from tqdm import tqdm
import phoenix as px# Define the evaluation model
eval_model = OpenAIModel(
azure_endpoint=AZURE_OPENAI_ENDPOINT,
azure_deployment="gpt-4o", # I'm using gpt-4o for evaluation
model=AZURE_OPENAI_CHAT_COMPLETION_DEPLOYED_MODEL_NAME,
api_key=AZURE_OPENAI_API_KEY,
api_version="2024-02-01"
)# Define evaluators
hallucination_evaluator = HallucinationEvaluator(eval_model)
qa_correctness_evaluator = QAEvaluator(eval_model)
relevance_evaluator = RelevanceEvaluator(eval_model)# List of project names
projects = ["Baseline", "Contextual"]# Create DataFrames for queries and retrieved documents for each project
queries_dfs = {}
retrieved_documents_dfs = {}for project in projects:
queries_dfs[project] = get_qa_with_reference(px.Client(endpoint="https://llamatrace.com/v1/traces"), project_name=project)
retrieved_documents_dfs[project] = get_retrieved_documents(px.Client(endpoint="https://llamatrace.com/v1/traces"), project_name=project)# Loop through each project and perform evaluations
for project in projects:
queries_df = queries_dfs[project]
retrieved_documents_df = retrieved_documents_dfs[project]
# Run evaluations
hallucination_eval_df, qa_correctness_eval_df = run_evals(
dataframe=queries_df,
evaluators=[hallucination_evaluator, qa_correctness_evaluator],
provide_explanation=True,
)
relevance_eval_df = run_evals(
dataframe=retrieved_documents_df,
evaluators=[relevance_evaluator],
provide_explanation=True,
)[0]
# Log evaluations
px.Client(endpoint="[https://llamatrace.com/v1/traces](https://llamatrace.com/v1/traces)").log_evaluations(
SpanEvaluations(eval_name=f"Hallucination_{project}", dataframe=hallucination_eval_df),
SpanEvaluations(eval_name=f"QA Correctness_{project}", dataframe=qa_correctness_eval_df),
DocumentEvaluations(eval_name=f"Relevance_{project}", dataframe=relevance_eval_df),
)LBKS```LBKSLBKSThis should resolve the connection issue by ensuring that all `px.Client` instances are correctly pointing to the remote Phoenix instance.LBKSSources:
馃挕聽Hint: Mention RunLLM in the thread for followups.
RunLLM i get this error now: HTTPStatusError: Redirect response '302 Found' for url 'https://llamatrace.com/v1/traces/v1/spans?project_name=Baseline&project-name=Baseline' Redirect location: '/login' For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/302
The HTTPStatusError with a 302 redirect indicates that the request is being redirected to a login page, which typically means that authentication is not properly set up. Ensure that you have set the PHOENIX_API_KEY environment variable correctly and that it is being used in the px.Client initialization.Here鈥檚 how you can modify your code to include the API key in the px.Client initialization: 1. Set the API Key Environment Variable: Ensure that the PHOENIX_API_KEY is set in your environment.
2. Pass the API Key in the Client Initialization: Use the api_key parameter when initializing px.Client.Here鈥檚 the updated code block:
HallucinationEvaluator,
OpenAIModel,
QAEvaluator,
RelevanceEvaluator,
run_evals,
)
from phoenix.session.evaluation import get_qa_with_reference, get_retrieved_documents
from phoenix.trace import DocumentEvaluations, SpanEvaluations
from tqdm import tqdm
import phoenix as px
import osLBKS# Ensure the PHOENIX_API_KEY environment variable is set
PHOENIX_API_KEY = os.getenv("PHOENIX_API_KEY")LBKS# Define the evaluation model
eval_model = OpenAIModel(
azure_endpoint=AZURE_OPENAI_ENDPOINT,
azure_deployment="gpt-4o", # I'm using gpt-4o for evaluation
model=AZURE_OPENAI_CHAT_COMPLETION_DEPLOYED_MODEL_NAME,
api_key=AZURE_OPENAI_API_KEY,
api_version="2024-02-01"
)LBKS# Define evaluators
hallucination_evaluator = HallucinationEvaluator(eval_model)
qa_correctness_evaluator = QAEvaluator(eval_model)
relevance_evaluator = RelevanceEvaluator(eval_model)LBKS# List of project names
projects = ["Baseline", "Contextual"]LBKS# Create DataFrames for queries and retrieved documents for each project
queries_dfs = {}
retrieved_documents_dfs = {}LBKSfor project in projects:
queries_dfs[project] = get_qa_with_reference(px.Client(endpoint="[https://llamatrace.com/v1/traces](https://llamatrace.com/v1/traces)", api_key=PHOENIX_API_KEY), project_name=project)
retrieved_documents_dfs[project] = get_retrieved_documents(px.Client(endpoint="[https://llamatrace.com/v1/traces](https://llamatrace.com/v1/traces)", api_key=PHOENIX_API_KEY), project_name=project)LBKS# Loop through each project and perform evaluations
for project in projects:
queries_df = queries_dfs[project]
retrieved_documents_df = retrieved_documents_dfs[project]
# Run evaluations
hallucination_eval_df, qa_correctness_eval_df = run_evals(
dataframe=queries_df,
evaluators=[hallucination_evaluator, qa_correctness_evaluator],
provide_explanation=True,
)
relevance_eval_df = run_evals(
dataframe=retrieved_documents_df,
evaluators=[relevance_evaluator],
provide_explanation=True,
)[0]
# Log evaluations
px.Client(endpoint="[https://llamatrace.com/v1/traces](https://llamatrace.com/v1/traces)", api_key=PHOENIX_API_KEY).log_evaluations(
SpanEvaluations(eval_name=f"Hallucination_{project}", dataframe=hallucination_eval_df),
SpanEvaluations(eval_name=f"QA Correctness_{project}", dataframe=qa_correctness_eval_df),
DocumentEvaluations(eval_name=f"Relevance_{project}", dataframe=relevance_eval_df),
)LBKS```LBKSLBKSEnsure that the `PHOENIX_API_KEY` environment variable is correctly set in your environment before running the script.LBKSSources:
- [Hosted Phoenix Documentation](https://docs.arize.com/phoenix/hosted-phoenix)
- [Authentication Setup](https://docs.arize.com/phoenix/setup/authentication#system-keys)
馃挕聽Hint: Mention RunLLM in the thread for followups.
still now working 馃槥 RunLLM 'https://llamatrace.com/v1/traces/v1/spans?project_name=Baseline&project-name=Baseline' Redirect location: '/login'
The persistent 302 redirect to the login page suggests that the API key might not be correctly set or recognized. Let's ensure that the API key is correctly included in the request headers.1. Verify API Key: Double-check that the PHOENIX_API_KEY environment variable is correctly set and contains the right API key.
2. Explicitly Set Headers: Ensure that the API key is explicitly included in the headers when initializing the px.Client.Here鈥檚 the updated code block with explicit header settings: ```from phoenix.evals import (
HallucinationEvaluator,
OpenAIModel,
QAEvaluator,
RelevanceEvaluator,
run_evals,
)
from phoenix.session.evaluation import get_qa_with_reference, get_retrieved_documents
from phoenix.trace import DocumentEvaluations, SpanEvaluations
from tqdm import tqdm
import phoenix as px
import os# Ensure the PHOENIX_API_KEY environment variable is set
PHOENIX_API_KEY = os.getenv("PHOENIX_API_KEY")# Define the evaluation model
eval_model = OpenAIModel(
azure_endpoint=AZURE_OPENAI_ENDPOINT,
azure_deployment="gpt-4o", # I'm using gpt-4o for evaluation
model=AZURE_OPENAI_CHAT_COMPLETION_DEPLOYED_MODEL_NAME,
api_key=AZURE_OPENAI_API_KEY,
api_version="2024-02-01"
)# Define evaluators
hallucination_evaluator = HallucinationEvaluator(eval_model)
qa_correctness_evaluator = QAEvaluator(eval_model)
relevance_evaluator = RelevanceEvaluator(eval_model)# List of project names
projects = ["Baseline", "Contextual"]# Create DataFrames for queries and retrieved documents for each project
queries_dfs = {}
retrieved_documents_dfs = {}for project in projects:
client = px.Client(endpoint="https://llamatrace.com/v1/traces", headers={"Authorization": f"Bearer {PHOENIX_API_KEY}"})
queries_dfs[project] = get_qa_with_reference(client, project_name=project)
retrieved_documents_dfs[project] = get_retrieved_documents(client, project_name=project)# Loop through each project and perform evaluations
for project in projects:
queries_df = queries_dfs[project]
retrieved_documents_df = retrieved_documents_dfs[project]
# Run evaluations
hallucination_eval_df, qa_correctness_eval_df = run_evals(
dataframe=queries_df,
evaluators=[hallucination_evaluator, qa_correctness_evaluator],
provide_explanation=True,
)
relevance_eval_df = run_evals(
dataframe=retrieved_documents_df,
evaluators=[relevance_evaluator],
provide_explanation=True,
)[0]
# Log evaluations
client = px.Client(endpoint="[https://llamatrace.com/v1/traces](https://llamatrace.com/v1/traces)", headers={"Authorization": f"Bearer {PHOENIX_API_KEY}"})
client.log_evaluations(
SpanEvaluations(eval_name=f"Hallucination_{project}", dataframe=hallucination_eval_df),
SpanEvaluations(eval_name=f"QA Correctness_{project}", dataframe=qa_correctness_eval_df),
DocumentEvaluations(eval_name=f"Relevance_{project}", dataframe=relevance_eval_df),
)LBKS```
Ensure that the PHOENIX_API_KEY is correctly set in your environment before running the script. This approach explicitly sets the authorization header, which should help in resolving the 302 redirect issue.Sources:
- Hosted Phoenix Documentation
- Authentication Setup
馃挕聽Hint: Mention RunLLM in the thread for followups.
RunLLM Am I doing something here when trying to log evaluations? import phoenix as px px_client = px.Client(endpoint="https://llamatrace.com/v1/traces", api_key=PHOENIX_API_KEY) phoenix_df = px_client.get_spans_dataframe(project_name="baseline")
The issue might be related to the endpoint URL or the way the API key is being passed. Let's ensure that the endpoint URL and the API key are correctly set up. Additionally, the px.Client initialization should include the correct headers for authentication.Here鈥檚 how you can modify your code to ensure proper authentication and endpoint setup: 1. Set the API Key Environment Variable: Ensure that the PHOENIX_API_KEY is set in your environment.
2. Pass the API Key in the Headers: Use the headers parameter when initializing px.Client.Here鈥檚 the updated code block: LBKSimport phoenix as px import osLBKS# Ensure the PHOENIX_API_KEY environment variable is set PHOENIX_API_KEY = os.getenv("PHOENIX_API_KEY")LBKS# Initialize the Phoenix client with the correct endpoint and headers px_client = px.Client(endpoint="[https://llamatrace.com/v1/traces](https://llamatrace.com/v1/traces)", headers={"Authorization": f"Bearer {PHOENIX_API_KEY}"})LBKS# Retrieve spans dataframe for the specified project phoenix_df = px_client.get_spans_dataframe(project_name="baseline")LBKS# Display the dataframe print(phoenix_df.head())LBKSEnsure that the PHOENIX_API_KEY environment variable is correctly set in your environment before running the script. This approach explicitly sets the authorization header, which should help in resolving the 302 redirect issue.If the issue persists, please verify the following:
- The PHOENIX_API_KEY is correct and has the necessary permissions.
- The endpoint URL [https://llamatrace.com/v1/traces](https://llamatrace.com/v1/traces) is correct and accessible.Sources:
- https://github.com/Arize-ai/phoenix/issues/4920
- https://arize-ai.slack.com/archives/C04R3GXC8HK/p1722266142.082829
馃挕聽Hint: Mention RunLLM in the thread for followups.
brilliant, ping me when
