HIPAA Compliant GPT: How to Set Up Using AWS Bedrock, Google Vertex AI, and Azure OpenAI

Key takeaways

• You can run a HIPAA compliant GPT today if you use cloud providers that sign a Business Associate Agreement (BAA).
• Top HIPAA-friendly platforms: AWS Bedrock, Google Vertex AI, and Azure OpenAI—each offers enterprise controls and data-use guarantees.
• Pricing is often comparable to direct vendor rates; expect small extra costs for networking, logging, and fine-tuning hosting.
• Follow a practical checklist: BAA, private networking, encryption (CMEK/KMS), strict IAM, audit logging, and PHI minimization.

Opening (hook + promise)

HIPAA compliant GPT does not require you to avoid GPT, Claude, or Gemini. You can run a HIPAA compliant GPT today.

Here’s the key: use cloud providers that sign a Business Associate Agreement (BAA) and offer enterprise-grade controls. That’s how you protect PHI, keep audit trails, and ensure your data isn’t used to train public models.

In this guide you’ll get:

• Which providers to use — AWS Bedrock, Google Vertex AI, Azure OpenAI
• Model options — Claude, GPT‑4, Gemini — and HIPAA-compliant AI posture
• Real pricing realities
• A practical setup checklist you can follow this week

Keep scrolling for the exact steps and tradeoffs that matter in the real world.

HIPAA basics for AI usage

HIPAA focuses on PHI data protection. For AI, that means:

• Safeguards: encryption, access controls, and breach response
• Data handling: limit who sees PHI and why; keep audit logs
• Accountability: prove what happened, when, and by whom

Why a Business Associate Agreement (BAA) matters:

• A BAA binds the provider to HIPAA rules
• It enforces proper PHI handling and breach duties
• It is the contract layer that makes HIPAA compliant LLMs possible at scale

Helpful context on HIPAA security expectations:

• Administrative, physical, and technical safeguards are required under HIPAA.

The three main HIPAA-friendly routes to top models

AWS Bedrock (HIPAA)

What you can use:

• Anthropic Claude (e.g., Sonnet, Opus)
• Meta Llama, Amazon Titan, and more

Why teams choose it:

• AWS signs a BAA; Bedrock is a HIPAA-eligible service
• Enterprise AI security: VPC endpoints via AWS PrivateLink, KMS encryption, IAM, CloudTrail
• Data protection: your inputs/outputs aren’t used to train foundation models

Where it shines: Fast access to the newest Claude models and strong PHI data protection controls out of the box.

Google Vertex AI (HIPAA)

What you can use: Gemini (Pro, Flash), select PaLM, and open-source models.

Why teams choose it:

• Google signs a BAA; Vertex AI is a HIPAA-eligible product
• Governance: VPC Service Controls, CMEK, IAM, audit logging, DLP patterns
• Data-use defaults: prompts and responses aren’t used to train Google models by default

Where it shines: Gemini for fast, cost-effective reasoning and tight integration with Google Cloud security.

Azure OpenAI (HIPAA)

What you can use: GPT‑4 family, GPT‑4 Turbo, DALL·E, and more.

Why teams choose it:

• Microsoft signs a BAA for in-scope services and Azure OpenAI provides enterprise-grade controls
• Private networking via Private Link, RBAC, Key Vault, Defender for Cloud (private networking)
• Data-use controls: customer prompts and completions are not used to train OpenAI models in Azure OpenAI

Where it shines: Organizations standardized on Microsoft security and easy policy enforcement with Azure Policy and logging.

Pricing reality check (cost is comparable to going direct)

Good news: HIPAA compliant GPT does not have to be pricey. In many cases, you’ll pay similar rates to going direct.

What we see in the field:

• Claude on Bedrock: pricing is essentially at Anthropic’s posted rates (Bedrock pricing, Anthropic pricing).
• OpenAI models on Azure: generally aligned with OpenAI’s published API pricing (Azure OpenAI pricing, OpenAI pricing).
• Gemini on Vertex AI: typically matches Google’s public rates for enterprise usage (Vertex AI pricing).

Extra costs to watch:

• Fine-tuned model hosting and training fees (watch Azure OpenAI hosting costs: Azure pricing).
• Egress/networking, logging, and key management across clouds.

Takeaway: With a BAA and enterprise controls, HIPAA compliant AI can be cost-parity with direct vendor APIs—without sacrificing PHI data protection.

Implementation experience and setup flow

If you’ve built on OpenAI/Anthropic/Google APIs, building on Bedrock, Vertex AI, or Azure OpenAI will feel familiar. The main difference is extra guardrails: auth, network, and logging.

What changes:

• Auth and identity: use IAM (AWS), IAM (GCP), or Entra ID/RBAC (Azure)
• Networking: private endpoints/VPC/VNet to keep traffic off the public internet
• Logging and keys: centralized audit logs and KMS/CMEK everywhere

Practical setup checklist

• Choose your provider(s) based on your primary models (Claude → AWS Bedrock, GPT‑4 → Azure OpenAI, Gemini → Vertex AI).
• Execute a Business Associate Agreement (HIPAA BAA for AI) with your cloud provider.
• Configure dedicated enterprise infrastructure:
  • Private endpoints: AWS PrivateLink, Google Private Service Connect/VPC SC, Azure Private Link.
  • Encryption: enforce TLS in transit and KMS/CMEK at rest.
  • Logging: enable audit logs (CloudTrail/CloudWatch, Cloud Logging, Azure Monitor).
• Lock down data-use settings:
  • Disable training or retention features by default.
  • Confirm provider statements on data non-use for training (AWS Bedrock data privacy, Vertex AI data governance, Azure OpenAI data privacy).
• Implement PHI minimization/redaction:
  • Drop identifiers you don’t need (name, MRN, SSN).
  • Use pattern-based redaction or de-identification before prompts.
  • Re-identify only on the client or secure service layer.
• Enforce least privilege and secret hygiene: fine-grained IAM, rotate keys, store secrets in KMS/Key Vault/Secret Manager.
• Document everything for audits: data flows, subprocessors, retention policy, access reviews, incident response, and model cards/use cases.

Tip: Think in layers: network isolation, encryption, identity, logging, and data-use controls. Each layer blocks a different risk. Together, they create robust enterprise AI security.

Access and approval timelines (what to expect)

Access isn’t hard, but timing varies by provider and account history.

What teams report:

• AWS Bedrock: often immediate once the service is enabled in your account/region.
• Google Vertex AI: usually available right away; some orgs see 1–2 business days for quota increases.
• Azure OpenAI: access requires approval; typical is ~1 business day, sometimes longer based on use case.

If you need day-one access to brand-new models, there are tradeoffs and workarounds. In the next sections we cover model availability timing, a medical transcription case study, and a quick-start guide you can run this week.

Tradeoffs vs. going direct to model vendors

Model availability timing

• New models don’t always land everywhere at once.
• AWS Bedrock often gets new Claude releases quickly; Gemini updates land in Vertex AI first; GPT‑4 family updates arrive in Azure OpenAI after OpenAI.com.
• Expect a lag from a few days to several weeks depending on provider and region.

When day-one access matters

If you need immediate access for research or feature testing, going direct to a model vendor may be faster — but direct APIs usually don’t include a BAA or full enterprise controls you need for PHI protection.

For production with PHI, the safer path is AWS Bedrock HIPAA, Google Vertex AI HIPAA, or Azure OpenAI HIPAA with a signed BAA and private networking.

Mitigations: get the best of both

• Run a multi-provider strategy: prototype on whichever service has the newest model, then move to your HIPAA-compliant stack before real PHI traffic.
• Keep a portable prompt and schema: use a consistent JSON output spec across providers.
• Build a thin adapter layer: one interface, many backends (Bedrock, Vertex, Azure).
• Lock in controls, not vendors: make network, IAM, logging, and DLP the foundation so you can swap models without reopening compliance work.

Real-world case study: HIPAA-compliant medical transcription app

Context

A multi-site medical group wanted fast, accurate clinical notes from visit audio. Strict PHI rules, detailed audit logs, and no training on customer data. Goals: clean transcripts, smart editing, and safe clinician chat.

Architecture choices

• Speech-to-text: existing ASR vendor output sent into secure cloud storage.
• Transcript cleanup and structure: Claude via AWS Bedrock for sectioning, grammar, and SOAP note formatting.
• Chat-based editing and Q&A: Gemini via Google Vertex AI for quick follow-ups and formatting tweaks.
• Why these picks: Claude quality on Bedrock and Gemini low-latency chat on Vertex (Bedrock data privacy, Vertex data governance).

Data flow (PHI-aware)

1. Upload audio and ASR text to a private bucket with CMEK/KMS encryption.
2. Run de-identification on obvious identifiers before LLM calls when possible.
3. Send batched, minimized text to Claude on Bedrock via PrivateLink.
4. Store LLM outputs with audit logs (CloudTrail/CloudWatch or Cloud Logging).
5. Provide an editor UI where staff ask Gemini for changes.
6. Re-identify only at the secure service layer, then export to EHR.

Security and governance

• Private networking end to end: AWS PrivateLink and Google Private Service Connect/VPC Service Controls (AWS PrivateLink, Google VPC SC).
• Keys in KMS/CMEK; strict IAM/RBAC roles; secrets in Key Vault/Secret Manager equivalents.
• Model data-use controls disabled by default; no training on customer data (Bedrock data privacy, Vertex governance).

Outcome

• Clinicians received cleaner drafts in seconds, with fewer edits.
• PHI stayed in HIPAA-eligible services under a Business Associate Agreement.
• Cost was near vendor direct rates, plus small spend for networking and logs.
• The team kept the option to add Azure OpenAI later for GPT‑4 features while keeping Azure OpenAI HIPAA guardrails (Azure data privacy).

Advanced options and extensibility

Host or customize models

• Bedrock supports multiple foundation models and enterprise controls; check HIPAA eligibility for any new capability before using PHI (AWS HIPAA reference).
• Vertex AI supports tuning and grounding with enterprise governance; align scopes with VPC Service Controls and DLP (Vertex governance).
• Azure OpenAI supports fine-tuning and model deployments with private networking and Key Vault integration (Azure private networking).

Fine-tuning within HIPAA constraints

• Use de-identified datasets for training when possible.
• Keep raw PHI in your VPC/VNet and apply strict access controls.
• Budget for fine-tune hosting and training costs, especially on Azure OpenAI (Azure pricing).

Observability and governance add‑ons

• Centralize logs: CloudTrail/CloudWatch, Cloud Logging, Azure Monitor.
• Add DLP and redaction at ingress and egress.
• Human review queues for sensitive outputs (e.g., discharge notes).
• Regular access reviews and incident runbooks to back your HIPAA compliant AI controls (HIPAA security guidance).

Quick-start guide: Make your GPT deployment HIPAA-compliant

• Decide your workloads: transcription cleanup, SOAP notes, patient summaries, chat, coding suggestions.
• Pick your models: Claude for structured clinical writing; GPT‑4 on Azure for broad reasoning; Gemini for fast chat.
• Choose providers: AWS Bedrock HIPAA for Claude; Google Vertex AI HIPAA for Gemini; Azure OpenAI HIPAA for GPT‑4.
• Execute your HIPAA BAA for AI: Ensure the services you’ll use are in scope under the BAA (AWS, Google, Microsoft).
• Set up enterprise AI security: Private endpoints (PrivateLink, Private Service Connect/VPC SC, Azure Private Link), TLS and KMS/CMEK, and audit every call.
• Lock down data-use: Confirm prompts and completions aren’t used to train models (AWS, Google, Azure).
• Minimize PHI: Redact unnecessary identifiers; re-identify only inside your secure app.
• Pilot and scale: Validate latency, cost, and quality; add rate limits, retries, and circuit breakers; document data flows and retention for audits.

FAQ

Are GPT or Claude HIPAA compliant by default?

No. The models themselves are not “HIPAA compliant” on their own. Compliance comes from how you deploy them: under a BAA, with enterprise controls, and with safeguards around PHI. Using HIPAA-eligible services like Bedrock, Vertex AI, or Azure OpenAI is the usual path.

Do OpenAI or Anthropic sign BAAs via standard APIs?

Most teams do not rely on direct vendor APIs for PHI because a BAA and enterprise controls are not typically available in standard self-serve plans. Instead, teams use cloud providers that sign a BAA and provide network isolation, IAM, and audit logging.

Will my PHI be used to train models?

On HIPAA-eligible cloud services, providers state that prompts and completions are not used to train foundation models. Always verify and disable any data retention features (AWS, Google, Azure).

Is running local LLMs safer than cloud?

It can be, but only if you match enterprise AI security: physical security, encryption, RBAC, patching, high availability, monitoring, and incident response. For most teams, HIPAA-eligible cloud services with a BAA are faster and safer to operate at scale (HIPAA security guidance).

What’s the cost difference between HIPAA compliant LLMs and direct APIs?

Often small to none. Azure OpenAI typically aligns with OpenAI pricing; Bedrock pricing for Anthropic models is similar to Anthropic direct; Vertex AI is close to Google’s public rates. Expect extra spend for networking, logging, and fine-tuned model hosting (Azure pricing, OpenAI pricing, Bedrock pricing, Anthropic pricing, Vertex pricing).

Can I use multiple cloud providers at once?

Yes. Many teams mix AWS Bedrock for Claude, Vertex AI for Gemini, and Azure OpenAI for GPT‑4. Build a small abstraction layer and keep prompts portable to avoid lock-in.

How long does it take to get access?

• Bedrock: often immediate after enabling the service (getting started).
• Vertex AI: usually immediate; quotas may take 1–2 business days (quotas).
• Azure OpenAI: approval is required; many teams see about one business day (Azure OpenAI access).

What controls matter most for PHI data protection?

Private networking, encryption with CMEK/KMS, strict IAM/RBAC, audit logs, and clear data-use settings that prevent training on your data. Add DLP and PHI minimization for defense in depth (HIPAA guidance).

Conclusion and next steps

You can ship HIPAA compliant GPT today. Use HIPAA-eligible services with a signed Business Associate Agreement, then layer network isolation, encryption, IAM, logging, and data-use controls. AWS Bedrock, Google Vertex AI, and Azure OpenAI give you top models—Claude, Gemini, and GPT‑4—without sacrificing PHI data protection.

A smart path: start where your must-have model lives, keep prompts portable, move production PHI to the cloud that gives you the BAA and controls you need, and revisit your mix as models and prices change.

If you want help standing this up, grab our checklist, subscribe for practical updates, or reach out. We’ll get your first HIPAA compliant AI workflow live this week—and your HIPAA compliant GPT stack ready for scale.