Who decided to proceed with HealthCare.gov despite the known security risks?

December 21st, 2013

When blogging about Obamacare on a daily basis, it is often difficult to keep perspective, as I juggle a million balls with every new development. But as I begin to think about the narrative for the sequel, Unraveled, I am starting to weave together different strands to tell the story of Healthcare.gov in a chronological fashion.

One of these threads focuses on the decision to launch the website even though people on the project knew it wasn’t ready–but in terms of operational capacity, and security. But, due to politics, the website went ahead anyway. Remarkably, all the brass from the President to Sebelius had plausible deniability, and claimed not to know about these problems. I find this implausible, but we can uncover that truth another day.

For now, my inquiry focuses on who actually approved these systems to launch. In a previous life, I worked in network security in the DOD. My knowledge is somewhat out of date, but I have a rough idea of the process by which federal information systems are approved for usage. It is a really strict process, that requires signatures from lots of government officials. I was involved in several projects, that were much lower priority, that were delayed many times because it could not get the proper certifications and accreditations. So what happened with HealthCare.gov?

Remember the CBS News Report about all of the slipped deadlines for security?

As HealthCare.gov was being developed, crucial tests to ensure the security and privacy of customer information fell behind schedule.

CBS News analysis found that the deadline for final security plans slipped three times from May 6 to July 16. Security assessments to be finished June 7 slid to August 16 and then August 23. The final, required top-to-bottom security tests never got done.

The House Oversight Committee released an Obama administration memo that shows four days before the launch, the government took an unusual step. It granted itself a waiver to launch the website with “a level of uncertainty … deemed as a high (security) risk.”

The AP reported on this memo:

An internal government memo obtained by The Associated Press shows administration officials were concerned that a lack of testing posed a “high” security risk for President Barack Obama’s new health insurance website.

The Sept. 27 memo to Medicare chief Marylin Tavenner said a website contractor wasn’t able to test all the security controls in one complete version of the system.

Insufficient testing “exposed a level of uncertainty that can be deemed as a high risk,” the memo said.

The memo recommended setting up a security team to address risks, conduct daily tests, and a full security test within two to three months of going live.

Trenkle, the guy who retired with no reason, wrote the memo. Despite this risk, Tavenner proceeded.

Agency head Marilyn Tavenner accepted the risk and “mitigation” measures like frequent testing and a dedicated security team. But three other officials signed a statement saying that “does not reduce the risk” of launching October 1.

The same Tavenner who had no idea the site wouldn’t work. And one of the risks could have disclosed account information through a password reset function. Another person logged in and saw someone else’s information.

Yet, Tavenner said:

“When consumers fill out the online application, they can trust that the information they’ve provided is protected by stringent security standards and that the technology underlying the application process has been tested and is secure,” Medicare administrator Marilyn Tavenner assured the Senate’s Health Committee on Tuesday.

But a short while later, Tavenner acknowledged the Carolinas security breach. “We actually were made aware of that” Monday, she said in response to a question from Sen. Johnny Isakson, R-Ga. “We implemented a software fix.”

It was not immediately clear how the North Carolina man was able to view the personal information of the man in South Carolina. However, a vulnerability that has afflicted websites for years is known as “horizontal privilege escalation,” in which a legitimate user of a website slightly alters the string of random-looking characters in the website’s address or inside downloaded data files known as “cookies,” causing the system to display information about the accounts of other users. It can be protected against by a well-designed website.

Further, Henry Chaio, the Chief Project Manager at CMS was also kept in the dark.

CBS News has learned that the project manager in charge of building the federal health care website was apparently kept in the dark about serious failures in the website’s security. Those failures could lead to identity theft among buying insurance. The project manager testified to congressional investigators behind closed doors, but CBS News has obtained the first look at a partial transcript of his testimony.

Henry Chao, HealthCare.gov’s chief project manager at the Centers for Medicare and Medicaid Services (CMS), gave nine hours of closed-door testimony to the House Oversight Committee in advance of this week’s hearing. In excerpts CBS News has obtained, Chao was asked about a memo that outlined important security risks discovered in the insurance system.

Chao said he was unaware of a Sept. 3 government memo written by another senior official at CMS. It found two high-risk issues, which are redacted for security reasons. The memo said “the threat and risk potential (to the system) is limitless.” The memo shows CMS gave deadlines of mid-2014 and early 2015 to address them.

But Chao testified he’d been told the opposite.

“What I recall is what the team told me, is that there were no high findings,” he said.

Chao testified security gaps could lead to identity theft, unauthorized access and misrouted data.

Based on this false information, Chao gave the goahead to launch:

It was Chao who recommended it was safe to launch the website Oct. 1. When shown the security risk memo, Chao said, “I just want to say that I haven’t seen this before.

A Republican staff lawyer asked, “Do you find it surprising that you haven’t seen this before?”

Chao replied, “Yeah … I mean, wouldn’t you be surprised if you were me?” He later added: “It is disturbing. I mean, I don’t deny that this is … a fairly nonstandard way” to proceed.

Sharyl Attkisson, who has been covering this issue closely for CBS, has a new report, titled “High security risk found after HealthCare.gov launch”

A top HealthCare.gov security officer told Congress there have been two, serious high-risk findings since the website’s launch, including one on Monday of this week, CBS News has learned.

Teresa Fryer, the chief information security officer for the Centers for Medicare and Medicaid Services (CMS), revealed the findings when she was interviewed Tuesday behind closed doors by House Oversight Committee officials. The security risks were not previously disclosed to members of Congress or the public. Obama administration officials have firmly insisted there’s no reason for any concern regarding the website’s security.

The Department of Health and Human Services (HHS) responded to questions about the security findings in a statement that said, “in one case, what was initially flagged as a high finding was proven to be false. In the other case, we identified a piece of software code that needed to be fixed and that fix is now in place. Since that time, the feature has been fully mitigated and verified by an independent security assessment, per standard practice.”

 What is more troubling is that Fryer was “overruled” by her superiors when she recommended shutting down the site.

In another security bombshell, Fryer told congressional interviewers that she explicitly recommended denial of the website’s Authority to Operate (ATO), but was overruled by her superiors. The website was rolled out amid warnings Fryer said she gave both verbally and in a briefing that disclosed “high risks” and possible exposure to “attacks”.

Fryer also said that she refused to put her name on a letter recommending a temporary ATO be granted for six months while the issues were sorted out.

“My recommendation was a denial of ATO,” Fryer told Democrats and Republicans who sat in on the day-long interview.

What happened when she recommend a denial of an ATO? She was overruled. And, the guy she told this to retired in November.

According to Fryer, she first recommended denying the ATO to CMS chief information officer Tony Trenkle based on the many outstanding security concerns after pre-launch testing.

“I had discussions with him on this and told him that my evaluation of this was a high risk,” Fryer told the committee. Trenkle retired from his CMS job on Nov. 13. He has not responded to CBS News interview requests.

Though Trenkle did not sign the ATO.


When pressed about this, Sebelius’s answer was non-committal, stressing that no one who reported to her advised a delay.

This is the first time a government insider has gone on record challenging the administration’s insistence that there were no worrisome security concerns. On Oct. 30, Rep. Gus Bilirakis, R-Fla., asked Health and Human Services (HHS) Secretary Kathleen Sebelius in testimony to Congress whether “any senior department officials” advised delaying the rollout of HealthCare.gov.

“I can tell you that no senior official reporting to me ever advised me that we should delay,” Sebelius answered. “We have testing that did not advise a delay. So not — not to my knowledge.”

 But Fryer’s testimony conflicts Sebelius’s account. She told the brass to delay the launch. She was ignored.

But Fryer says she briefed Sebelius’ top information officers at HHS in a teleconference on Sept. 20, recommending the website’s launch be delayed for security reasons. Fryer testified that the call included HealthCare.gov’s chief project manager Henry Chao, HHS chief information security officer Kevin Charest and HHS Deputy Assistant Secretary for Information Technology Officer Frank Baitman. Fryer says she learned three days later that her advice was not going to be followed.

Fryer also testified that she took part in preparing a Sept. 23 briefing for CMS Chief Operating Officer Michelle Snyder. Fryer’s contribution to the briefing, a slideshow presentation, outlined multiple “high risks,” “risk of unknown” and “risk of attacks.” She told the House Oversight Committee that her concerns arose after security testing discovered “uncertainties” and “unknown risks.”

However, Fryer testified that “unknown risks” can’t be remediated or mitigated.

Fryer told congressional officials that besides the new high risks exposed, there have also been new “moderate” security risk findings as well as a couple of new “low” findings.

And what about Trenkle, the guy who retired? He didn’t sign off on the launch.

Tony Trenkle, the Obamacare official in charge of HealthCare.gov security effortsannounced his resignation Wednesday, effective next week.

CBS News has learned that Trenkle, the Chief Information Officer for the Centers for Medicare and Medicaid Services (CMS), was originally supposed to sign off on security for the glitch-ridden website before its Oct. 1 launch, but didn’t. Instead, the authorization on September 27 was given by Trenkle’s boss, CMS administrator Marilyn Tavenner.

As CBS News reported Monday, security assessments fell behind and the website never had the required top-to-bottom tests.

Trenkle and two other CMS officials, including Chief Operating Officer Michelle Snyder, signed an unusual “risk acknowledgement” saying that the agency’s mitigation plan for rigorous monitoring and ongoing tests did “not reduce the (security) risk to the … system itself going into operation on October 1, 2013.”

Of course Sebelius was not aware of these waivers.

Both Democrats and Republicans have raised security concerns in two days of Senate hearings. Wednesday, Health and Human Services Secretary Kathleen Sebelius told Congress she did not know about the special security waiver that her agency head, Tavenner, granted the website.

“I was not aware of this and I did not have these discussions with the White House because I wasn’t aware of them,” Sebelius testified.

Sen. Richard Burr, R-N.C., asked, “Did the White House know there had been no end-to-end testing of the security aspects of the exchange?”

“I think the White House was aware of operational issues involving end-to-end testing and I – I don’t know of the specifics of – again, I did not have the discussions about this authority to operate issue with the White House,” said Sebelius.

When Tavenner was asked about the security authorization she agreed she would sign off on it.

Sen. Pat Roberts, R-Kansas, asked Tavenner Tuesday about the website’s unusual security authorization without the required testing.

“Are you the official at CMS responsible for making…the security authorization decisions?” Roberts asked.

Tavenner replied, “So I think in the case, because of the visibility of the exchange, the Chief Information Officer wanted to make me aware of it and I agreed to sign it with their recommendation to proceed.”

Wednesday, an HHS spokesman said that the reason Tavenner, not Trenkle, signed the security authorization is because HealthCare.gov is “a high-profile project and CMS felt it warranted having the administrator sign the authority to operate memo.” HHS also says there is an aggressive risk mitigation plan in effect, “the privacy and security of consumers personal information is a top priority for us” and personal information is “protected by stringent security standards.”

Yahoo News describes that mitigation plan:

According to federal law and policy, all government computer systems must have a security certification before going live.

Tavenner approved the Sept. 27 security certification for the health website, which read: “Aspects of the system that were not tested due to the ongoing development exposed a level of uncertainty that can be deemed as a high risk.”

It called for a four-step mitigation plan, including ongoing monitoring and testing, leading to a full security control assessment.

The agency’s top three information security professionals signed on an accompanying page that said that “the mitigation plan does not reduce the risk to the … system itself going into operation on Oct. 1” but that its added protections would reduce risk later and ensure full testing within six months.

And of course Tavenner said nothing to her boss:

HealthCare.gov has two major components: an electronic “back room” that did get full security certification and the consumer-facing “front room” that’s temporarily certified.

The back room, known as the federal data hub, pings government agencies to verify applicants’ personal information. It does not store data.

But the front room does. That’s where consumers in the 36 states served by the federal website create and save their accounts. While the individual components of the front room did undergo security testing, the system as a whole could not be tested because it was being worked on until late in the game.

Tavenner testified that was the reason she had to issue a temporary certification. The decision was brought to her level because of the overall magnitude of the project, she said. She said she didn’t voice the security concerns to her boss, Health and Human Services Secretary Kathleen Sebelius, or to the White House office that oversees federal agencies.

Allahpundit opines here:

 Fryer’s not the only techie at CMS whose signature was mysteriously missing from the Authority to Operate. Remember Tony Trenkle? He was the project manager who left the agency in November — an unusual move given the all-hands-on-deck attitude to fixing Healthcare.gov at the time. Trenkle also didn’t sign the ATO. Ispeculated at the time that he refused for the same reasons that Fryer did, namely, that no tech specialist with a conscience would greenlight a site this vulnerable, but the official explanation was that CMS chief Marilyn Tavenner wanted to sign the ATO herself because this project was super-important and should be formally endorsed by the head of the agencyor whatever. Sure looks like Tavenner was fully aware of how dangerous Healthcare.gov could be to users who entered their private information but insisted that the site be launched anyway, over the objections of her own team. It’s subpoena time.

This is to say nothing about the ability to launch the site even though it didn’t work. Recall that in March, 2013 Henry Chao, the CIO, warned that he was “nervous” about the launch, and hoped it wouldn’t be a “third-world experience.”

“The time for debating about the size of text on the screen or the color or is it a world-class user experience, that’s what we used to talk about two years ago,” Henry Chao, an official at the Centers for Medicaid and Medicare Services who is overseeing the technology of the exchangessaid at a recent conference. “Let’s just make sure it’s not a third-world experience.”

Chao also described himself as “nervous.” His comments, which came at a policy meeting of insurance industry lobbying group America’s Health Insurance Plans, were first reported by CQ Health Beat and picked up by Avik Roy at Forbes.

I see grand jury investigations and perhaps indictments in the future for some involved in this launch.