ShieldForce Colored Variance Logo
Is ChatGPT HIPAA Compliant? A Home Health and Hospice Guide to Shadow AI
Shadow AI

Is ChatGPT HIPAA Compliant? A Home Health and Hospice Guide to Shadow AI

9 min read
SF
Obi Ibeto

Not the way most staff use it. A personal ChatGPT account, free or paid, is not covered by a HIPAA business associate agreement. When a home health nurse or hospice...

Not the way most staff use it. A personal ChatGPT account, free or paid, is not covered by a HIPAA business associate agreement. When a home health nurse or hospice care coordinator pastes patient details into one, that is an impermissible disclosure your agency must investigate as a possible breach. A written AI policy, an approved tool, training and monitoring prevent it.

This is no longer hypothetical. In early November 2026, HIPAA Journal reported two separate cases of patient information entered into AI tools without authorization:

  • At Lake Burien Transitional Care Facility in Washington State, a staff member used a personal ChatGPT account to draft summaries of client case notes, including names, diagnoses and medications. The state found one instance, then discovered the employee had done it several times before. It wrote to 12 families. A similar incident affected four patients at Western State Hospital.

  • At Weill Cornell Medicine in New York, patient data from a radiology research study was entered into AI and machine-learning tools against the organization's own policy. The data included medical record numbers, imaging details, diagnoses and medications.

Neither case involved a hacker. Both involved well-meaning people trying to get through their work faster. That is exactly the situation in most home health and hospice agencies today, where clinicians and coordinators write visit notes, care summaries and referral letters all day, often from a phone between patient homes.

This guide explains where the HIPAA line is, what to do if staff have already crossed it, and how to give your team the benefits of AI without putting patient data at risk.

Is ChatGPT HIPAA compliant?

No AI tool is "HIPAA compliant" on its own. What matters is whether your agency has a signed business associate agreement with the company behind it, and has set the tool up safely. Without that agreement, patient information should never go into it.

A business associate agreement (BAA) is the contract in which a vendor commits to protect patient data the way HIPAA requires. Any company that creates, receives, stores or transmits protected health information (PHI) for you counts as a business associate. That includes your EHR, your email provider and any AI service your staff type patient details into.

The U.S. Department of Health and Human Services (HHS) is direct about cloud services. Its Guidance on HIPAA and Cloud Computing says a cloud provider that stores or processes electronic PHI is a business associate, even if it only holds encrypted data. It also says an organization that uses such a service without a BAA is violating the HIPAA Rules. An online AI tool works the same way: what staff type in is sent to and processed on the vendor's servers.

So where does ChatGPT fall?

Version

Can it be covered by a BAA?

Free or paid personal accounts

Not listed by OpenAI as eligible. Treat as off-limits for patient information.

Specific business offerings, such as ChatGPT for Healthcare or ChatGPT Enterprise with a Regulated Workspace

Listed by OpenAI as available with a BAA, under conditions and for listed features only.

OpenAI's API (used by software developers)

A BAA may be available for eligible services. This is how some vendors build HIPAA-ready tools on OpenAI's models.

Source: OpenAI's help center page on HIPAA-eligible products and functionality, which also says features not listed are not automatically covered. Vendors change these terms, so confirm the current version before relying on it.

The same logic applies to Google Gemini, Microsoft Copilot, Claude and every other AI assistant. A well-known brand name does not create a BAA. Your agency's signed agreement, and how the tool is configured, are what count.

What happens if a nurse or care coordinator puts patient information into ChatGPT?

Under HIPAA, it is presumed to be a reportable breach unless your agency can document a low probability that the information was compromised. The work starts the moment someone notices.

The HIPAA Breach Notification Rule treats an impermissible disclosure of PHI as a breach unless a documented risk assessment shows otherwise. That assessment weighs four factors:

  1. What was disclosed. A first name and visit time is very different from a full name, diagnosis, medication list and Social Security number.

  2. Who received it. A consumer AI service with no BAA is an unauthorized recipient.

  3. Whether it was actually acquired or viewed. Once text is submitted, it has left your control and sits on the vendor's servers.

  4. How far the risk was reduced. For example, whether the chat history and account were deleted and the vendor's data-retention terms.

If the assessment can't show a low probability of compromise, notification rules apply:

  • Patients: notify each affected person without unreasonable delay, and no later than 60 days after discovery.

  • HHS: within 60 days if 500 or more people are affected. Smaller breaches go in an annual report.

  • Media: required when more than 500 residents of one state are affected.

  • States: many states add their own notice rules on top of HIPAA.

What to do in the first 48 hours

  • Stop it. Tell the staff member to stop using the tool for any patient work. Don't shame them; you need their full cooperation to learn what was shared.

  • Find out what went in. Which patients, which details, and how many times. In the Washington case, the first discovery led to several earlier uses.

  • Reduce the exposure. Have the staff member delete the conversations and, if appropriate, the account. Record what was deleted and when.

  • Document everything. Who, what, when, and every decision you make. This record is what HHS will ask to see.

  • Run the four-factor assessment with your privacy officer, and get legal advice before deciding not to notify.

  • Fix the cause. One person's shortcut usually means others are doing the same thing. Treat it as a gap in your program, not just one employee's mistake.

For the full post-incident timeline, see What Happens in the First 30 Days After a Home Health Data Breach and our HIPAA breach notification guide.

How can home health and hospice agencies stop shadow AI without banning AI?

Give staff a safe way to get the help they're already looking for, then close the unsafe ways. A ban on its own pushes the behavior onto personal phones, where you can't see it.

Shadow AI means AI tools staff use for work without the organization's review or approval. It is common. A Wolters Kluwer Health survey, cited by HIPAA Journal, found 40% of hospital workers knew of colleagues using unauthorized AI tools, and 20% admitted using them. Home health and hospice staff work more independently than hospital staff, so there's little reason to think the rate is lower.

These five steps map to requirements already in the HIPAA Security Rule.

1. Find out what's already in use

Ask staff directly, without blame, which AI tools they use and for what: visit-note cleanup, care plan wording, referral letters, family emails, scheduling. Check browser extensions and apps on agency devices. Then add AI tools to your annual risk analysis, the documented review of where patient data lives and what could go wrong.

HIPAA tie-in: risk analysis, 45 CFR 164.308(a)(1)(ii)(A). Our step-by-step HIPAA risk analysis guide walks through the process.

Need Audit-Ready HIPAA Compliance for Your Field Team?

Protect remote clinical staff, field laptops, mobile EVV apps, and EHR systems starting at $35/user/month.

2. Offer an approved tool, and make the rule simple

The rule is one line: no patient information goes into any tool unless the agency has a signed BAA for it. Then give staff an approved option for the tasks they want help with. That could be an AI feature inside your EHR, a business AI service with a BAA, or a vetted vendor tool. If an approved tool isn't available yet, tell staff which tasks are fine without patient details, like drafting a general policy or a training handout.

HIPAA tie-in: business associate contracts, 164.308(b)(1) and 164.502(e). See our vendor security assessment and BAA guide.

3. Train every role, including field and per diem staff

Show real examples: a care coordinator pasting a discharge summary into a chatbot to shorten it, or a nurse dictating visit notes into an AI app on a personal phone. Explain why it matters in plain terms: once submitted, the data is on someone else's servers. Pair the training with a written sanctions policy, applied consistently.

HIPAA tie-in: security awareness and training, 164.308(a)(5); sanctions policy, 164.308(a)(1)(ii)(C). Our annual security training guide covers format and documentation.

4. Add technical guardrails

On agency-managed laptops and phones, block or restrict consumer AI sites and apps for roles that handle patient data. Data loss prevention (DLP) tools, which spot and stop sensitive information leaving a device, can catch patient identifiers before they're pasted into a web form. Review activity logs for traffic to AI services.

HIPAA tie-in: information system activity review, 164.308(a)(1)(ii)(D); audit controls, 164.312(b). If staff use personal devices, see Personal Devices on Home Wi-Fi.

5. Treat an AI slip as a security incident

Put the AI scenario in your incident response plan: who staff tell, who checks what was shared, who runs the breach risk assessment, and who decides on notification. Make reporting easy and safe, so staff come forward early instead of hoping no one notices.

HIPAA tie-in: security incident procedures, 164.308(a)(6). Start with our incident response plan template.

What should a HIPAA AI policy for a home health agency include?

A good AI policy fits on one or two pages and answers the questions a field nurse would actually ask. Long policies go unread; short, specific ones get followed.

At a minimum, cover these points:

Policy element

What to write down

Approved tools

The exact tools staff may use with patient information, each with a signed BAA on file.

The bright line

No patient information in any tool not on the approved list, including on personal phones and accounts.

Allowed without PHI

Tasks that are fine with any tool when no patient details are included, such as drafting a general handout.

Accounts

Work AI use only through agency-managed accounts, never personal ones.

Human review

A clinician reviews and owns anything AI drafts for a patient record or a family.

Requesting a new tool

Who staff ask, and how the agency checks security and gets a BAA before approval.

Reporting mistakes

Who to tell if patient data went into the wrong tool, how fast, and an assurance that early reporting is encouraged.

Sanctions

What happens for violations, consistent with your existing sanctions policy.

Training and sign-off

Covered at onboarding and in annual training, with each person's acknowledgment kept on file.

Review date

When the policy is updated. AI tools and vendor terms change quickly.

You can fold this into your acceptable use policy rather than creating a separate document. Our guide to writing a HIPAA-compliant acceptable use policy shows how. For record-keeping once AI tools are approved, see HIPAA and AI Documentation: What Home Health Agencies Must Record in 2026.

This article is general information, not legal advice. Have your attorney review your policy and any breach decision.

The bottom line

Your staff are already using AI because it saves them time, and they're right that it can. The risk isn't AI itself. It's patient information going into tools your agency has no agreement with and no view into. Set one clear rule, give staff an approved option, train everyone, and make it safe to report mistakes. That turns AI from a quiet liability into something your agency can use with confidence.

ShieldForce works alongside whatever platform you already run, whether that's Homecare Homebase, WellSky, MatrixCare or Axxess. We add the monitoring, data loss prevention, training and incident response that sit outside the EHR, at a flat $35 per user per month.

Find the guide for your organization:

Organization

Compliance self-assessment

Home health

Home Health Compliance Audit

Hospice

Hospice Compliance Audit

Palliative care

Palliative Care Compliance Audit

FQHC / community health center

FQHC Compliance Audit

Sources

This article is for informational purposes only and does not constitute legal advice. Consult qualified HIPAA counsel for guidance specific to your organization.

Share this post

Need Audit-Ready HIPAA Compliance for Your Field Team?

Protect remote clinical staff, field laptops, mobile EVV apps, and EHR systems starting at $35/user/month.

Topics

#Shadow AI#business associate agreement#HIPAA compliance#AI Security#Home health#Hospice
Free Security Assessment

Ready to Secure Your Business?

Don't let cyber threats put your business at risk. Discover how ShieldForce protects organizations like yours - 24/7.