Remote Notebooks: Current Workarounds and Future Support Update

Mikyo · 2023-11-14T17:06:41.351Z

Hey @Kumar M. so we don't have built-in support for remote notebooks quite yet. We have some ongoing work to properly proxy all network requests to the running thread (cc @Amy X.) but for now the only solution is to setup a proxy via something like ngrok (https://docs.arize.com/phoenix/reference/frequently-asked-questions#can-i-use-phoenix-locally-from-a-remote-jupyter-instance). Here's the outstanding ticket (https://github.com/Arize-ai/phoenix/issues/675) feel free to upvote but it's definitely scheduled! Let us know if ngrok or some other proxy works for you for now.

Remote Notebooks: Current Workarounds and Future Support Update | Arize AI Community

14 comments

· Sorted by Oldest

Kumar M.
·
Thank You Mikyo. I will try to get ngrok work in AWS. Just from a production deployment perspective, all AI/ML will be done in a cloud environment and would be helpful to use phoenix without all this proxy complexity. Will keep you posted.
Mikyo
·
Totally understand. Will work on getting this over the line.
Kumar M.
·
Quick question. Can I launch phoenix on an AWS EC2 instance and not the notebook to act as a central callback system. I can expose the EC2 instance to the internet
Kumar M.
·
on a web server like NGINX
Kumar M.
·
Best case scenarios for multiple AI engineers working on multiple LLM apps: Install phoenix on a central EC2 or Container for tracking all LLM Applications. Write to this central phoenix server from each notebook using a unique application id.
Kumar M.
·
Any help here from your product/support team would be great. I can get on a call as well as this is time sensitive for me. Thanks a bunch
Kumar M.
·
Looks like we can run on a central host as per the api documentation at https://docs.arize.com/phoenix/api/session
Kumar M.
·
def launch_app( primary: Optional[Dataset] = None, reference: Optional[Dataset] = None, corpus: Optional[Dataset] = None, trace: Optional[TraceDataset] = None, host: Optional[str] = None, port: Optional[int] = None, run_in_thread: Optional[bool] = True, ) -> Session
Mikyo
·
hey Kumar M. you certainly can figure out how to run the phoenix collector on a central phoenix instance. But as you mention there's no notion of things like workspaces with phoenix. We are working to support persistence in phoenix but it might not be a perfect solution right now for self-hosting. Alternatively you can export and write your traces to remote cloud storage and view them via Phoenix when you need to. Happy to intro you to the Arize product/support team if you'd like.
Kumar M.
·
An intro would be great as it would significantly reduce my time to research and try out things that might not work
Kumar M.
·
my email is kmantripragada@raptorsdigital.com
Kumar M.
·
One last question and I am done. Can you point me to any articles on “Alternatively you can export and write your traces to remote cloud storage and view them via Phoenix when you need to.”
Mikyo
·
Yup, so phoenix sessions can always be resumed on previous data so if you want to take a snapshot of your data, you can simply write the files to disc and share these assets.
from phoenix.trace.span_json_encoder import spans_to_jsonl with open("trace.jsonl", "w") as f: f.write(spans_to_jsonl(callback_handler._tracer.span_buffer))
You can always pass this data back into the launch session to view them. You can also hook into the callback handlers and persist the spans to disk on a periodic basis as well.
Mikyo
·
FYI Kumar M. since this is an OSS project we don't really have product/support but if you are looking for a full fledged enterprise platform, I can put you in touch with with our solutions team there. For phoenix our dev team will support you the best we can.
👍1

Kumar M.
·
Thank You Mikyo. I will try to get ngrok work in AWS. Just from a production deployment perspective, all AI/ML will be done in a cloud environment and would be helpful to use phoenix without all this proxy complexity. Will keep you posted.
Mikyo
·
Totally understand. Will work on getting this over the line.
Kumar M.
·
Quick question. Can I launch phoenix on an AWS EC2 instance and not the notebook to act as a central callback system. I can expose the EC2 instance to the internet
Kumar M.
·
on a web server like NGINX
Kumar M.
·
Best case scenarios for multiple AI engineers working on multiple LLM apps: Install phoenix on a central EC2 or Container for tracking all LLM Applications. Write to this central phoenix server from each notebook using a unique application id.
Kumar M.
·
Any help here from your product/support team would be great. I can get on a call as well as this is time sensitive for me. Thanks a bunch
Kumar M.
·
Looks like we can run on a central host as per the api documentation at https://docs.arize.com/phoenix/api/session
Kumar M.
·
def launch_app( primary: Optional[Dataset] = None, reference: Optional[Dataset] = None, corpus: Optional[Dataset] = None, trace: Optional[TraceDataset] = None, host: Optional[str] = None, port: Optional[int] = None, run_in_thread: Optional[bool] = True, ) -> Session
Mikyo
·
hey Kumar M. you certainly can figure out how to run the phoenix collector on a central phoenix instance. But as you mention there's no notion of things like workspaces with phoenix. We are working to support persistence in phoenix but it might not be a perfect solution right now for self-hosting. Alternatively you can export and write your traces to remote cloud storage and view them via Phoenix when you need to. Happy to intro you to the Arize product/support team if you'd like.
Kumar M.
·
An intro would be great as it would significantly reduce my time to research and try out things that might not work
Kumar M.
·
my email is kmantripragada@raptorsdigital.com
Kumar M.
·
One last question and I am done. Can you point me to any articles on “Alternatively you can export and write your traces to remote cloud storage and view them via Phoenix when you need to.”
Mikyo
·
Yup, so phoenix sessions can always be resumed on previous data so if you want to take a snapshot of your data, you can simply write the files to disc and share these assets.
from phoenix.trace.span_json_encoder import spans_to_jsonl with open("trace.jsonl", "w") as f: f.write(spans_to_jsonl(callback_handler._tracer.span_buffer))
You can always pass this data back into the launch session to view them. You can also hook into the callback handlers and persist the spans to disk on a periodic basis as well.
Mikyo
·
FYI Kumar M. since this is an OSS project we don't really have product/support but if you are looking for a full fledged enterprise platform, I can put you in touch with with our solutions team there. For phoenix our dev team will support you the best we can.
👍1