In the early days of the AI boom, health systems often jumped at the chance to try out the latest tool.
Many organizations ended up with a surplus of redundant — or ineffective — solutions. But they discovered key insights that they’ve applied to later AI investments, IT leaders told Becker’s.
“One of the biggest lessons from our early AI work is that a promising tool is not a good enough reason to start. We learned that the hard way,” said Muhammad Siddiqui, CIO of Richmond, Ind.-based Reid Health. “Like a lot of organizations, we moved toward ideas that sounded interesting before we had fully defined the problem we were trying to solve.”
Now, his health system starts with the operational need and outcome it wants before looking at any AI solution, gathering clinical input and establishing governance early, not after the fact. The organization tracks success post-go-live with the same rigor it used to research the application.
“It has made us slower to start and faster to get real results,” Mr. Siddiqui said.
Salt Lake City-based Intermountain Health learned that operational ownership matters more than technical sophistication, said Ryan Smith, senior vice president and chief digital and information officer.
“We’ve repeatedly seen that AI initiatives deliver the most value when there is clear sponsorship, defined success metrics, and early integration into real clinical and operational workflows,” he said.
Alignment extends beyond executive sponsors to the front-line caregivers who use the tools, Mr. Smith said. Senior leaders must set clear priorities and measures of success, engaging clinicians early to design the application to fit into their daily work.
“When that shared ownership exists, even complex AI solutions become more adoptable, trusted, and impactful for patients and caregivers,” he said.
Like many organizations, Boston Children’s Hospital acted quickly to partner with specialized AI vendors before fully testing whether their capabilities were different from what the major enterprise platforms could offer. “We were eager to innovate, and that urgency sometimes outpaced our diligence,” said John Brownstein, PhD, senior vice president and chief innovation officer of Boston Children’s.
But the experience taught an important lesson: Before committing to a big contract with an outside vendor, the health system now conducts thorough comparisons across competing approaches, including platforms that it’s already deploying, such as OpenAI.
“It forces us to ask hard questions: Is this vendor solving a problem that truly requires specialized expertise, or are we paying a premium for something a foundation model already handles well?” Dr. Brownstein said. “That discipline has saved us resources, reduced complexity in our AI stack, and honestly made us better partners to the vendors we do choose to work with.”
Not all IT leaders want a second chance with the technology. “I do not have any regrets about past AI decisions,” said Nallan Sriraman, chief technology officer of Somerville, Mass.-based Mass General Brigham. “That includes stopping ungoverned AI pilots, being deliberate in putting a broader AI governance framework in place and taking an iterative approach to AI overall — adjusting as we learn.”
Rather than moving too quickly on AI, Jacksonville, Fla.-based Baptist Health didn’t execute swiftly enough, said Aaron Miri, DHA, executive vice president and chief digital and information officer of Jacksonville, Fla.-based Baptist Health.
“We approached AI thoughtfully, but in hindsight, we overindexed on caution instead of accelerating where we had clear use cases,” he said.
The health system has pivoted to a more assertive stance with partners, expecting codevelopment, quick iteration, and explicit accountability around results — not just pilots.
“The biggest lesson is that AI in healthcare can’t be funded on ‘hope,’” Dr. Miri said. “Every initiative needs a defined ROI, whether that’s measurable time returned to clinicians or a direct financial impact. If you can’t tie it to the bedside or the balance sheet, it shouldn’t move forward.”
The post AI regrets? Health systems learn lessons from the early boom appeared first on Becker's Hospital Review | Healthcare News & Analysis.
Source: Read Original Article
