Are any lawyers accessing LLMs through the API and a third-party UI?
Throughout last year, I started using ChatGPT Plus more for work--so much that I upgraded to Pro a few months ago and loved it. It works great for anything in the public record or where data security isn't an issue. But I also have some work where data/cyber security is an issue, and I've just become more wary of where data is going, where it's being stored, who can access it, etc. I'm primarily a solo; I consult with law firms and work with corporate clients, but I've read that I'd have a hard time implementing any type of ZDR protocol or BAA with the major LLM providers, as that's usually reserved for their large enterprise users.
I don't have a programming background, but I went down a deep rabbit hole a couple of weeks ago researching alternatives that give me the functionality of something like ChatGPT Pro with more "enterprise"-grade security protocols that small businesses would have trouble implementing directly with the LLM providers (ZDR, HIPAA compliance, encryption, etc.).
That research led me to UIs like LibreChat, TypingMind, and a few others, along with other online tools that I had no idea even existed (like I said, I'm a lawyer, not a programmer), like OpenRouter, AWS Bedrock, Cloudflare/S3-compatible sync/backup, RAG, and a laundry list of plugins, extensions, and thingamabobs that I'm still navigating through.
My ultimate goal was to have a setup where data (chats, prompts, inputs, documents, etc.) are stored only with a secure provider with encryption, that is HIPAA-compliant, that doesn't involve third-party access, and that could sync across devices--MacBook Pro, iPhone, and iPad. I finally have something that's been working pretty well, although I'm still learning more and more every week about all of the things I can potentially build out.
Are any other lawyers experimenting with "custom" setups like this? If so, what are you implementing, and how are you using it?