Consider a hypothetical: a practice manager uses an AI coding assistant to build a prior authorization tracker over a weekend. By Monday, three front-desk staff rely on it. Six months later, nobody is sure where its data lives, who can change it, or what happens if the builder leaves. In a September 2026 MGMA poll, about a third of medical groups said they had built a tool in-house with AI. Practices adopting these tools should also establish proper AI governance for healthcare to manage security, accountability, and responsible AI use.
The short answer: Staff-built AI tools can be worth keeping, but only once the practice treats them as software it owns. That means an inventory of what exists, a risk tier for each tool based on what data it touches, a named owner, a business associate agreement (BAA) wherever ePHI flows to a vendor, and basic engineering hygiene: access control, testing, backups and monitoring. Tools that cannot meet that bar should be rebuilt properly or retired. A structured healthcare cybersecurity and compliance approach can help practices manage these risks.
How Common Are Staff-Built AI Tools in Medical Practices?
More common than many owners realize. In a September 22, 2026 MGMA Stat poll, published September 23, about 34% of medical group leaders said their organization had built its own app or workflow tool in-house using AI. About 8% had tried but not launched one, and about 10% were unsure. MGMA suggests the unsure group raises a governance question in itself, since leaders may not know what is being built, though the poll cannot confirm this.
Physicians want safeguards first. In the AMA's 2026 physician survey, published March 12, 2026, 86% named data privacy as a requirement before adopting AI tools, and 88% named safety and efficacy validation. Those are exactly the controls a weekend build tends to skip.
Why Do In-House AI Tools Become Risky After Launch?
Building the first version is now the easy part. The risk sits in everything that comes after it. A tool built in a few days often skips the work that makes software safe to depend on:
- Data goes places nobody mapped: Prompts, logs, error reports and test copies can all hold patient data. MGMA's guidance warns practices not to assume a consumer AI account includes a BAA.
- Vendors become business associates: HHS guidance (last reviewed December 23, 2022) states that a cloud service provider storing ePHI is a business associate even if it holds only encrypted data and lacks the key. Storing ePHI there without a BAA creates a HIPAA compliance gap.
- AI-generated code is still code: It needs authentication, permissions, patching and testing. MGMA cautions that AI assistants can suggest software packages that do not exist, which attackers can then register.
- Ownership is unclear: Who holds the source code, passwords and API keys? MGMA advises putting this in writing, especially when a physician-owner or contractor built the tool, because ownership "is not automatic."
- Clinical features raise the stakes: A tool that recommends care may be a regulated device. FDA's Clinical Decision Support Software guidance, issued January 29, 2026, says one test is whether a clinician can independently review the basis for the recommendation.
None of this means staff should stop experimenting. But a prototype people rely on has become a production system.
How Should a Practice Triage the Tools It Already Has?
If you have not yet inventoried the AI tools in your practice, our guide to writing an AI policy for a medical practice covers that step. This section focuses on tools staff built themselves. Place each on a risk ladder based on what it can see and do. MGMA suggests a similar ladder; this version adds what each tier should require.
| Risk tier | What the tool does | Example | Minimum before continued use |
|---|---|---|---|
| 1. No PHI | Works only on public or internal non-patient data | Staff scheduling helper, policy search | Named owner; listed in inventory |
| 2. Reads PHI | Views patient data but changes nothing | Referral status dashboard | BAA for every vendor in the data path; role-based access; added to the HIPAA risk analysis |
| 3. Writes to systems | Updates EHR, PM or payer portals | Auto-filling prior auth forms | Everything above, plus testing, audit logs and a rollback plan |
| 4. Creates clinical content | Drafts notes or patient letters | Patient letter drafter | Everything above, plus defined human review before anything reaches a patient or chart |
| 5. Recommends care | Suggests diagnosis or treatment | Risk-scoring tool | Everything above, plus clinical validation and an FDA status review |
Reassess the tier whenever someone adds a feature. A read-only dashboard that gains a "send" button has moved up the ladder.
What Does a Supportable In-House Tool Need?
Use this checklist for any tool at tier 2 or above. It turns a working prototype into something the practice can run, fix and defend.
- A data-flow map: Every place data goes: prompts, model provider, database, logs, backups, analytics and error reporting.
- Signed BAAs with every vendor that creates, receives, maintains or transmits ePHI in that flow.
- Real authentication and roles: Individual logins, least-privilege access and immediate removal when staff leave.
- Source code in a managed repository the practice controls, with credentials stored outside the code.
- Dependency checks: Every package verified as real and maintained, with a patching schedule.
- Tests on realistic cases, using synthetic or de-identified data, including what happens when an input is wrong or a service fails.
- Audit logs showing who viewed or changed what, kept long enough for your record-retention rules.
- Backups and a fallback: A tested restore, and a manual process staff can return to.
- A named owner and a backup owner, plus a written change-approval step.
- Monitoring: Basic alerts for errors, unusual access and model output that drifts from expectations.
- An insurance check: MGMA advises not to assume cyber or professional liability cover extends to in-house software.
Illustrative scenario: a five-provider orthopedic group finds four staff-built tools during its inventory. Two are tier 1 and simply get owners. The prior auth tracker is tier 3 and runs on a personal cloud account without a BAA. The group moves it to a practice-owned environment, adds logins and logs, and keeps the old spreadsheet running until testing is done. A patient letter drafter is paused until a clinician review step is built in.
Should You Keep, Harden, Rebuild or Retire a Tool?
Not every tool deserves rescue. MGMA's advice is to "price ownership, not creation": the lasting costs are maintenance, security, integrations, testing, monitoring, infrastructure and staff support time.
| Option | When it fits | Watch out for |
|---|---|---|
| Keep as is | Tier 1, low use, clear owner | Scope creep into PHI |
| Harden | Valuable tool with a sound design but missing controls | Hidden dependencies on one person |
| Rebuild properly | Valuable workflow, but the prototype cannot be secured or integrated | Treating the rebuild as a quick fix |
| Replace with a product | A commercial or EHR feature now does the same job | Losing the workflow detail staff designed |
| Retire | Low value, high risk, or no owner | Staff reverting to unsanctioned workarounds |
Integration often decides the answer. A tool that copies data by hand between the EHR, practice management system and payer portals will break when any of them changes. A properly built version uses supported interfaces and APIs.
What Are the Risks and Implementation Notes?
Don't drive tools underground: A blanket ban pushes staff to consumer chatbots. Offer a lightweight intake review instead. As MGMA noted on January 21, 2026, "If someone can buy or enable an AI tool without review, you don't have governance."
Keep records retrievable: If a tool creates or changes part of the medical record, plan for access requests, amendments and legal holds.
Test for wrong answers, not only broken buttons: For tier 4 and 5 tools, review output against realistic cases with a clinician.
This is general information, not legal or regulatory advice. Confirm HIPAA, FDA and state-law questions with qualified counsel.
How Softnotions Can Help
The ideas behind most staff-built tools are good, because the people closest to the workflow designed them. Our job is to make those tools dependable without losing what makes them useful.
- Inventory and risk review: We help map each tool's data flow, risk tier and gaps against your AI governance approach.
- Hardening or rebuilding: Our healthcare product engineering teams add authentication, logging, testing and supported integrations, or rebuild the tool on a maintainable base.
- Security alignment: We bring the tool into your existing healthcare cybersecurity and compliance controls.
Through Forward Deployed Engineering, our engineers work inside your real environment with the staff who built and use the tool, so the knowledge does not stay with one person.
Staff-built AI tools show where your workflows hurt most. The goal is not to stop them but to give each one an owner, a risk tier and the engineering basics a production system needs, before a vendor change, a departure or an audit forces the question.
If you have tools in use that nobody fully owns, talk to our team and we will help you sort what to keep, harden or rebuild.


