WSJ "There's a Medical App for That—Or Not" - Misinformation on Health IT Safety Regulation?

There's a health IT meme that just won't die (patients may, but not the meme).

It's the meme that health IT "certification" is a certification of safety.

I expressed concern about the term "certification" being misunderstood even before the meme formally appeared, when the term was adopted by HHS with regard to evaluation of health IT for adherence to the "meaningful use" pre-flight features checklist.  See my mid-2009 post "CCHIT Has Company" where I observed:

HIT "certification." ... is a term I put in quotes since it really is "features qualification" at this point, not certification such as a physician receives after passing Specialty Boards.

The "features qualification" is an assurance that the EHR functions in way that could enable an eligible provider or eligible hospital to meet the Center for Medicare & Medicaid Services' (CMS) requirements of "Meaningful Use."  No rigorous safety testing in any meaningful sense is done, and no testing under real-world conditions is done at all.

I've seen the meme in various publications and venues.  I've even seen it in legal documents in medical malpractice cases where EHR's were involved, as an attempted defense.

Now the WSJ has fallen for the health IT Certification meme.

An article "There's a Medical App for That—Or Not" was published on May 29, 2012.  Its theme is special regulatory accommodation for health IT in the form of opposition to FDA regulation of devices such as "portable health records and programs that let doctors and patients keep track of data on iPads."

In the article, this assertion about health IT "certification" is made:

... The FDA's approach to health-information technology risks snuffing out activity at a critical frontier of health care. Poor, slow regulation would encourage programmers to move on, leaving health care to roil away for yet another generation, fragmented, disconnected and choking on paperwork.

The process already exists for safeguarding the public for computers in health care. It's not FDA premarket review but the health information technology certification program, established under President George W. Bush and still working fine under the Obama Health and Human Services Department. The government sets the standards and an independent nonprofit [ATCB, i.e., ONC Authorized Testing and Certification Bodies - ed.] ensures that apps meet those standards. It's a regulatory process as nimble as the breakout industry it's meant to monitor. That is where and how these apps should be regulated.

It's a wonderful meme.  Unfortunately, it's wrong.  Dead wrong.

Certification by an ATCB does not "safeguard the public."   Two ONC Authorized Testing and Certification Bodies (ATCB's) admitted this in email, as in my Feb. 2012 post "Hospitals and Doctors Use Health IT at Their Own Risk - Even if Certified".  I had asked them, point-blank:

"Is EHR certification by an ATCB a certification of EHR safety, effectiveness, and a legal indemnification, i.e., certifying freedom from liability for EHR use of clinical users or organizations? Or does it signify less than that?"

I received two replies from major ONC ATCB's indicating that "certification" is merely assurance that HIT meets a minimal set of "meaningful use" guidelines, not that it's been vetted for safety.  For instance:

From: Joani Hughes (Drummond Group)
Sent: Monday, March 05, 2012 1:06 PM
To: Scot Silverstein
Subject: RE: EHR certification question

Per our testing team:

It is less than that. It does not address indemnification although a certification could be used as a conditional part of some other form of indemnification function, such as a waiver or TOA, but that is ultimately out of the scope of the certification itself. Certification in this sense is an assurance that the EHR functions in way that could enable an eligible provider or eligible hospital to meet the CMS requirements of Meaningful Use Stage 1. Or to restate it more directly, CMS is expecting eligible providers or eligible hospitals to use their EHR in “meaningful way” quantified by various quantitative measure metrics and eligible providers or eligible hospitals can only be assured they can do this if they obtain a certified EHR technology.

Please let me know if you have any questions.

Thank you,
Joani.

Joani Hughes
Client Services Coordinator
Drummond Group Inc.

The other ATCB, ICSA Labs, stated that:

... Certification by an ATCB signifies that the product or system tested has the capabilities to meet specific criteria published by NIST and approved by the Office of the National Coordinator. In this case the criteria are designed to support providers and hospitals achieve "Meaningful Use." A subset of the criteria deal with the security and patient privacy capabilities of the system.

Here is a list of the specific criteria involved in our testing:
http://healthcare.nist.gov/use_testing/effective_requirements.html

In a nutshell, ONC-ATCB Certification deals with testing the capabilities of a system, some of them relate to patient safety, privacy and security functions (audit logging, encryption, emergency access, etc.).

What was suggested in the email below (freedom from liability for users of the system, etc.) would be out of scope for ONC-ATCB testing based on the given criteria. [I.e., certification criteria - ed.] I hope that helps to answer your question.

I had noted that:

... My question was certainly answered [by the ATCB responses]. ONC certification is not a safety validation, such as in a document from NASA on aerospace software safety certification, "Certification Processes for Safety-Critical and Mission-Critical Aerospace Software" (PDF) which specifies at pg. 6-7:

In order to meet most regulatory guidelines, developers must build a safety case as a means of documenting the safety justification of a system. The safety case is a record of all safety activities associated with a system throughout its life. Items contained in a safety case include the following:

• Description of the system/software
• Evidence of competence of personnel involved in development of safety-critical software and any
safety activity
• Specification of safety requirements
• Results of hazard and risk analysis
• Details of risk reduction techniques employed
• Results of design analysis showing that the system design meets all required safety targets
• Verification and validation strategy
• Results of all verification and validation activities
• Records of safety reviews
• Records of any incidents which occur throughout the life of the system
• Records of all changes to the system and justification of its continued safety

A CCHIT ATCB juror, a physician informatics specialist, has also done a guest post in Jan. 2012 on HC Renewal about the certification process, reproducing his testimony to HHS on the issue.  That post is "Interesting HIT Testimony to HHS Standards Committee, Jan. 11, 2011, by Dr. Monteith."  Dr. Monteith testified (emphases mine):

... I’m “pro-HIT.” For all intents and purposes, I haven’t handwritten a prescription since 1999.

That said and with all due respect to the capable people who have worked hard to try to improve health care through HIT, here’s my frank message:

ONC’s strategy has put the cart before the horse. HIT is not ready for widespread implementation. 

... ONC has promoted HIT as if there are clear evidence-based products and processes supporting widespread HIT implementation.

But what’s clear is that we are experimenting…with lives, privacy and careers.

... I have documented scores of error types with our certified EHR, and literally hundreds of EHR-generated errors, including consistently incorrect diagnoses, ambiguous eRxs, etc.

As a CCHIT Juror, I’ve seen an inadequate process. Don’t get me wrong, the problem is not CCHIT. The problem stems from MU.

EHRs are being certified even though they take 20 minutes to do a simple task that should take about 20 seconds to do in the field.  [Which can contribute to mistakes and "use error" - ed.] Certification is an “open book” test. How can so many do so poorly?

For example, our EHR is certified, even though it cannot generate eRxs from within the EHR, as required by MU.

To CCHIT’s credit, our EHR vendor did not pass certification. Sadly, our vendor went to another certification body, and now they’re certified.

MU does not address many important issues. Usability has received little more than lip-service. What about safety problems and reporting safety problems? What about computer generated alerts, almost all of which are known to be ignored or overridden (usually for good reason)?

 

The concept of “unintended consequences” comes to mind.

All that said, the problem really isn’t MU and its gross shortcomings, it is ONC trying to do the impossible:

ONC is trying to artificially force a cure for cancer, basically trying to promote one into being, when in fact we need to let one evolve through an evidence-based, disciplined process of scientific discovery and the marketplace.

Needless to say, as was learned at great cost in past decades, a "disciplined process" in medicine includes meaningful safety regulation by objective outside experts.

Further, the certifiers have no authority to do important things such as forcibly remove dangerous software from the market.  An example is the forced Class 1 recall of a defective system as I wrote about in my Dec. 2011 post "FDA Recalls Draeger Health IT Device Because This Product May Cause Serious Adverse Health Consequences, Including Death".   Class 1 recalls are the most serious type of recall and involve situations in which there is a reasonable probability that use of these products will cause serious adverse health consequences or death.

In that situation, the producer had been simply advising users (in critical care environments, no less) to "work around the defects" that could indicate incorrect recommended dosage values of critical meds, including a drug dosage up to ten times the indicated dosage, as well as corrupt critical cardiovascular monitoring data.  As I observed:

... I find a software company advising clinicians to make sure to "work around" blatant IT defects in "acute care environments" the height of arrogance and contempt for patient safety.

Without formal regulatory authority to take actions such as this FDA recall, "safeguarding the public" is a meaningless platitude.

It's also likely the ATCB's, which are private businesses, would not want the responsibility of "safeguarding the public."  That responsibility would open them up to litigation when patient injuries or death were caused, or were contributed to, by "certified" health IT.

I have in the past also noted that the use of the term "certification" might have been deliberate, to mislead potential buyers exactly into thinking that "certification" is akin to a UL certification of an electrical appliance for safety, or an FAA approval of a new aircraft's flight-worthiness.

The WSJ needs to clarify and/or retract its statement, as the statement is misinformation.

At my Feb. 2012 post "Health IT Ddulites and Disregard for the Rights of Others" I observed:

Ddulites [HIT hyper-enthusiasts - ed.] ... ignore the downsides (patient harms) of health IT.

This is despite being already aware of, or informed of patient harms, even by reputable sources such as FDA (Internal FDA memo on H-IT risks), The Joint Commission (Sentinel Events Alert on health IT), the NHS (Examples of potential harm presented by health software - Annex A starting at p. 38), and the ECRI Institute (Top ten healthcare technology risks), to name just a few.

In fact, the hyper-enthusiastic health IT technophiles will go out of their way to incorrectly dismiss risk management-valuable case reports as "anecdotes" not worthy of consideration (see "Anecdotes and medicine" essay at this link).

They will also make unsubstantiated, often hysterical-sounding claims that health IT systems are necessary to, or simply will "transform" (into what, exactly, is usually left a mystery) or even "revolutionize" medicine (whatever that means).

Health IT is a potentially dangerous technology.   It requires meaningful regulation to "safeguard the public."  How many incidents like this and this will it take before that is understood by the hyper-enthusiasts?

I've emailed the ATCB's that had responded to my aforementioned query for clarification on the WSJ assertion about their role, being that the statement is in contradiction to their earlier replies to me.  I also advised them of the potential liability issues.

However, if it turns out to be true that the ONC-ATCB's do intend themselves as the ultimate watchdog and assurer of public safety related to EHR's, that needs to be known by the public and their representatives.

-- SS

Wall Street Journal Defends Hired WellPoint Executives' Lack of Accountability to the Company's Owners

The lack of accountability of the hired managers (or executives or bureaucrats) of health care organizations came into sharper focus thanks to a bizarre, in my humble opinion, Wall Street Journal editorial from last week. 

Background: Shareholder Campaign for Oversight of Hired Executives Use of Corporate Money for Political Purposes

In the background is the campaign by some of the owners, that is, shareholders of giant publicly held for-profit insurance company WellPoint to make its executives' attempts to involve the company in politics more transparent and accountable.  (See our previous post here.)  As noted more recently in Fortune (by way of CNN),

shareholders and major U.S. companies have been meeting behind the scenes to discuss improvements in oversight and disclosure practices. 'Companies need to remember that shareholders have a right to know how their money is being spent,' wrote Eric Sumberg, spokesperson for New York State Comptroller Thomas P. DiNapoli, representing the New York State pension fund, in an email. 'Transparency and full disclosure will help to deter high risk political spending that could hurt shareholder value.'

Aetna and WellPoint are two companies contending with shareholder proposals on political spending disclosure this year.

The Center for Public Accountability (CPA) rates the disclosures at Aetna and WellPoint as having 'room for improvement.' Both WellPoint and Aetna have disclosure practices that 'leave significant room for serious misrepresentation of the company's political spending through trade associations,' according to the Center's Political Accountability and Transparency Reports. According to the Center reports, both companies gave money to AHIP (American Health Insurance Plans). And $86 million in funds from AHIP were allegedly funneled to the Chamber of Commerce to lobby against health care reform, according to reports from Bloomberg and the National Journal.

Note that this money was supposedly used by WellPoint executives to undermine the Obama administration's health care reform proposals while the company was publicly supporting aspects of these proposals.

The Wall Street Journal Says Hired Executives Not Accountable to Shareholders

The Wall Street Journal's editorial page's denunciation of this campaign by corporate owners to assert their rights, and the accountability of hired managers opened thus,

The campaign to intimidate companies from exercising their free-speech rights is in high gear as shareholder proxy season arrives, and the most prominent early target is health-insurer WellPoint. The arc of this attack will be one of the election year's political leitmotifs, and it should be on the radar of every corporate boardroom.

In the favored new tactic of the left, unions and activists are using politicized shareholder resolutions to send a message to corporations: Drop support for free-market and conservative causes, or you'll take a political beating.

The Journal conveniently ignored that the campaign is not from outside the corporation, but from its very owners, and that the people they are supposedly trying to intimidate are actually supposed to be responsible to them.  In addition, it begged the question of how political spending by hired corporate bureaucrats unaccountable to the people who own the company could possible have anything to do with free markets.

If some owners do not think that executives should be spending company money on political causes (especially presumably causes that the executives favor, or that reflect the executives' self-interest), they have a perfect right to think so, and to act on their thoughts.


Then the  Journal went on to assail the shareholders' challenge to some members of the WellPoint board of directors.  After first defining Change to Win as a "union front group," -

Change to Win is now targeting WellPoint's annual meeting on May 16 when it will demand that shareholders vote against board members Julie Hill and Susan Bayh (wife of former Indiana Democratic Senator Evan Bayh) because the company has refused to disclose or stop all of its political spending. Among the company's crimes? Corporate funding of, you guessed it, ALEC.

Now let us back up a minute. This is about a campaign by stockholders, that is, people who are owners, albeit fractional owners of WellPoint. It is some shareholders who want to vote against the particular board members.  WellPoint directors are supposed to have a fiduciary duty to represent the stockholders', that is, the owners' financial interests. If stockholders think members of the board of directors are not representing the stockholders' interests, the stockholders have a perfect right to vote against them. 

However, the Journal fulminated,

The union attack on WellPoint is notable for targeting two board members by name and the effort to make extra hay out of Susan Bayh's political profile. (Added frisson: Evan Bayh has worked as a consultant to the Chamber.) The ad hominem attack is right out of the Saul Alinsky playbook and is intended as a warning to other corporate directors that their personal reputation will be damaged if they don't force companies to stop donating to industry groups.

Note further that all stockholders are owners, whether they are also union members, or have green hair. Note further that the owners again have a perfect right to criticize or vote against board members who they believe are not properly exercising their fiduciary responsibilities to stockholders, that doing so has nothing to do with the ad hominem fallacy, and that this right is not nullified for stockholders with particular political opinions, or stockholders whom the Wall Street Journal does not like.

Summary

So we see the Wall Street Journal, supposed defender of capitalism, attacking a fundamental part of capitalism, the right of ownership, corporate ownership in this case. Instead, presumably, the Journal editorialists thinks that hired corporate executives ought to be completely unaccountable to the stockholders, and able to do whatever they want, including to do what is in their self-interest but not the owners' interests.

So this is how far the coup d'etat by hired executives/ managers/ bureaucrats has progressed. Supposed defenders of capitalism are now defending the rule of hired corporate insiders, completely disregarding the rights of owners. All we are lacking is a catchy name for rule by the hired managers/ bureaucrats/ executives. I am open to suggestions.

We have long criticized leaders of health care organizations who are ill-informed, unaware or hostile to health care professionals' core values, self-interested, or even corrupt.  We have discussed how bad leadership has advanced as leaders have become less accountable.  It appears that the lack of accountability of health care leaders, and their tendencies to put their own interests first, is part of a larger problem.  This is the take-over by most of society's important organizations by the managers, bureaucrats, and executives who were hired to run them.  For profit corporate hired leaders have become unaccountable to the corporations' owners.  Non-profit organizations' hired leaders have become unaccountable for the mission, or for their organizations' stakeholders. 

If we want health care, and democratic society to survive, we need to counter the managers' coup d'etat and make leaders accountable once again. 

Dr. Silverstein and Dr. Poses in WSJ: "The Literature Is Hardly Pristine"

I have considered Dr. Roy Poses' Dec. 14, 2010 post "The Lancet Emphasizes the Threats to the Academic Medical Mission" (with its hyperlinks to source posts and articles) an excellent summary of many of the pathologies we address at Healthcare Renewal, especially with regard to the academic mission and the disruption of the integrity of the medical literature by commercial interests. His post is consistent with what might be considered our mission statement:

Addressing threats to health care's core values, especially those stemming from concentration and abuse of power. Advocating for accountability, integrity, transparency, honesty and ethics in leadership and governance of health care.

The Wall Street Journal published the following letter to the editor authored by me today in which I cited and summarized Dr. Poses' accounting of the medical literature's ills.

Unfortunately, the print version cannot contain the hyperlinks as in the aforementioned source post, but I have included them in the reproduced letter below in [brackets]. They are worth reviewing, along with additional links at the aforementioned source post "The Lancet Emphasizes the Threats to the Academic Medical Mission":

April 8, 2011
Wall Street Journal
Letters to the Editor

The Literature Is Hardly Pristine

I find it unfortunate having to inform reader James Reichmann, (Letters, April 1) who prefers his physician to recommend only treatments proven in the "synthesized medical literature," that the very literature on which he wishes his life to depend may be tainted.

As Dr. Roy Poses points out on the Healthcare Renewal Blog, numerous factors common in today's culture can and do corrupt the literature.

The factors include but are not limited to: rampant commercialization of medicine [here, here, here and here], research universities with lax conflict of interest policies [here], faculty as de facto employees of industry through grants [here], academics paid to be "key opinion leaders" to stealth-market drugs and devices [here], control of clinical research given to commercial sponsors [here], conflicts of interest allowing manipulation and suppression of clinical research [here and here], academics taking credit for articles written by commercially paid industry "ghost writers," [here and here], whistleblowing discouraged [here and here], leadership of academic medical centers by business people [here] and medical school leaders becoming stewards (as members of boards of directors) of for-profit health-care corporations [here, here, here and here].

As for me, until the medical literature can be freed of these contaminants, I'd rather trust a well-trained personal physician's good judgment in my own medical care.

Scot Silverstein, M.D.

Drexel University

Philadelphia

I believe it's also worth reviewing my own views on the subject, that the degree of contamination of medical literature is unknown and may be unrecoverable, due to spread of the contaminant vectors to the "experts" who then propagate the disease.

See my Aug. 2009 post "Has Ghostwriting Infected The Experts With Tainted Knowledge, Creating Vectors for Further Spread and Mutation of the Scientific Knowledge Base?" where I raise these questions.

At that post I suggest that while the damage might not be easily recoverable, the practices that lead to contaminated literature must be stopped going forward if true evidence-based medicine is ever to be a reality.

-- SS

4/12/2011 addendum:

Almost predictably, some anonymous person, this time over at the Respectful Insolence blog in commenting on a post there that attacks "naturopathic" medicine, proffered strawman arguments about my piece above in comment #26.

... So Dr. Silverstein thinks that the defects he cites (many of which have been revealed by and hotly debated within the scientific community) are ample justification for tossing out the entirety of research in favor of the sort of "clinical intuition" that's repeatedly been found false over the years

This binary, two-dimensional thinking is quite sad to read, if real. I'll be charitable in saying that the comment does have the "feel" of something contrived, such as received before from corporate sockpuppets.

-- SS

4/13 addendum:

The proprietor of Respectful Insolence assures me that "anonymous" quoted above is not a sockpuppet. My response was basically that before attacking non-anonymous authors by name on the web who in fact (as is clear from my writing here) share similar views on medical quackery, one should contact them first to ensure you truly understand their views.

(In my case, my view is that the "well-trained physicians" of good judgment I wrote of consider the literature critically but are not bound to it, in consideration of the unknown level of its commercialization-based contamination and the variability of individual patient situations. They treat the patient, not the guideline.)

-- SS

MedInformaticsMD in WSJ again: No Patient Will Ever Say, 'Quick, Watson, the Needle!'

Following along the lines of my Jan. 2011 post IBM's Watson, Jeopardy, and "Revolutionizing Medicine" , the Wall Street Journal published this Letter to the Editor by me today:

Wall Street Journal

LETTERS
FEBRUARY 25, 2011

No Patient Will Ever Say, 'Quick, Watson, the Needle!'

Regarding Ray Kurzweil's "When Computers Beat Humans on Jeopardy" (op-ed, Feb. 17): Librarians often receive requests for information as cryptic as "Jeopardy!" clues, from people who are sometimes not even sure what they're seeking. Watson is, in essence, a librarian that retrieves facts.

Regarding natural language processing and fact-retrieval systems like IBM's Watson, medicine is about cognition. It's about human judgment born of experience in dealing with ambiguity, not just of language but also, and this is critical, of observations, findings, lab data, image interpretation, etc. It is about human intuition, assemblage and the integration of a huge amount of disparate information in ways not well understood even by its practitioners. The end result is not just the recall of a piece of information, obscure as the information might be.

I consider predictions of Watson spearheading cybernetic miracles in medicine to imply just as grandiose a valuation to the technology as the statements I heard two decades ago about the health information technology of the day, or even today, "revolutionizing medicine." That has not happened.

A cybernetic librarian is no physician. As for me, for now I'll stick to people in my own medical care. Let Mr. Kurzweil see the machine for his ailments.

Scot Silverstein, M.D.

Philadelphia

I am pleased that my counter-marketing-hype opinion on the meme of health IT "revolutionizing medicine" was deemed fit to print by the Journal.

Those in senior leadership positions, as are included in the WSJ's readership, need to see that meme challenged.

-- SS