EP 70 – Zero Days and High Stakes: The US Treasury Attack

David Puner: You’re listening to the Trust Issues Podcast. I’m David Puner, a Senior Editorial Manager at CyberArk, the global leader in identity security.

December has become something of a notorious month for major cyberattacks. In December 2020, there was the landmark SolarWinds attack. The following December, attackers exploited a critical vulnerability in a widely used open-source software development library called Log4j, also known as Log4Shell. Then in 2022, it was the disclosure of the LastPass data breach.

Obviously, these are just a few prominent examples. Is it coincidental that these major attacks occurred or came to light during or near the holiday season? Probably not.

According to today’s guests, we’re here to discuss yet another high-profile attack, the details of which started to unfold last month—another unwelcome holiday cyber surprise. This time, an attack on the U.S. Treasury Department along with 16 other BeyondTrust customers, as we’ve learned in an investigation update published by BeyondTrust since we recorded this podcast conversation.

In this episode, Andy Thompson, CyberArk Labs Senior Offensive Research Evangelist, and Joe Garcia, CyberArk’s Principal DevOps Solutions Engineer, discuss the U.S. Treasury attack timeline, details, and potential reasons behind it. They also provide proactive security recommendations and insights into broader implications and future threat preparedness.

Quick note: We’re not here to criticize BeyondTrust. Our intention is to learn from this incident, to bolster our collective defenses against future cyber threats, and, in some small way, perhaps help prevent or mitigate the next potential major security event.

So let’s get right to it. Here’s my conversation with Andy Thompson and Joe Garcia.

David Puner: Andy Thompson, Senior Offensive Research Evangelist at CyberArk Labs, and Joe Garcia, DevOps Principal Solutions Engineer at CyberArk, welcome to the show, guys.

Andy Thompson: Good to be back.

Joe Garcia: And hello, Joe. I’ve been here this whole time. You never asked me to come on. No, I’m just kidding. I’m very thankful you asked me, and I appreciate being here to be able to hang out with Andy and you.

David Puner: Today, we’re here to discuss yet another major cyber breach. This time, it’s the recent attack on the U.S. Treasury Department, and you guys have been digging deeply into it.

So, to start things off and to give listeners an idea of what this breach is and what happened, Andy, first off, how does CyberArk Labs approach this kind of cybersecurity research and analysis? And then, Joe, we’ll talk to you about your angle on all of it as well.

Andy Thompson: The question is, how did this happen? Through multiple zero-day vulnerabilities? Something that, for all intents and purposes, was unavoidable? How did that cause a breach? You know, that’s something we could talk about.

David Puner: So you guys are thinking like attackers, which we all do here at CyberArk, but CyberArk Labs—that is the mindset, right?

Andy Thompson: Our marching orders are to think like an attacker so that Joe here can become a better defender—understanding the tactics, techniques, and procedures of what a threat actor did.

And again, we’re basing it only purely on publicly available information currently. But once we have this information in hand, Joe can provide us with better guidance as to how to protect against the attacks that we’re seeing in the wild.

David Puner: So, Joe, this is how you approach it. And this is how you and Andy have gotten together on this particular breach.

Joe Garcia: Yeah, absolutely. Andy has done a great job of really breaking down the timeline of events of this attack and providing me with all of the details of what transpired.

So really, it’s a great opportunity for me now to adjust my messaging to the different organizations that I engage with on a day-to-day basis—to start ensuring that I am providing them with the different defensive strategies and mitigation strategies they can employ so that this doesn’t happen to them now that we know the zero-days, how they work, and how they can be used in the real world.

David Puner: And I think that’s a great thing to mention at the top of this conversation, which is zero-days.

So for those who may not entirely know, what is a zero-day vulnerability?

Joe Garcia: If you think in terms of, like, the pandemic we just went through—it’s not like it goes from one day of nothing happening to now we’re in a pandemic. There’s always a patient zero—it has to start somewhere.

And in terms of vulnerabilities and exploits, a zero-day is a vulnerability or an exploit that has not been utilized in the wild before, so it’s the first time everyone is seeing it. Meaning, there is no defense for this particular exploit—similar to a patient zero.

So in this case, our patient zero is what we’re going to be talking about today in terms of the exploits that were utilized. But Andy, if you want to add to that, go for it.

Andy Thompson: Well, this isn’t just a regular zero-day. Zero-days come in all shapes, forms, and flavors, but this was an RCE—a remote code execution vulnerability. An unauthenticated code injection vulnerability that allowed really anybody from any distance to inject code into an application server.

And not just any application server—an application server that is touching government infrastructure.

This was a pretty nasty remote code execution vulnerability that we’re talking about. And that’s what makes this particular breach so interesting. Typically, we see end-days or just patched vulnerabilities being exploited by random organizations and nation-states too. But you rarely see zero-days in the wild.

So that’s what really brought our attention to this issue.

David Puner: So we’re sort of talking around what actually happened. Maybe we should get right into the details.

What do we know about this U.S. Treasury Department breach? When did we start getting visibility into it?

Andy Thompson: Visibility started in early December. That’s when they detected some sort of anomalous behavior. Several days later, they confirmed it was malicious in intent.

At that point, they started investigating it as a security incident, bringing in the victim—in this case, the U.S. Treasury Department—making them aware that, yes, they had been compromised.

They didn’t know how at first, just that a threat actor had accessed their systems in early December. But at least a week later, they determined that two separate zero-days had been exploited—the first was an unauthenticated remote code execution vulnerability, and then, a few days later, a second zero-day command injection vulnerability.

And that’s really where we lose visibility into the attack. This is where the speculation zone comes in—how the API key was acquired, which allowed the threat actor to pivot from BeyondTrust’s infrastructure into the infrastructure of the Treasury Department.

David Puner: So how did the attackers manage to obtain the API key and get into the Treasury Department, which is obviously a very high-profile target?

Joe Garcia: Unfortunately, that’s something we don’t know today.

All I can really speak to is the most common things we experience when we engage with organizations facing similar challenges.

Andy Thompson: That’s exactly why I wanted to bring you in, Joe. You spend a lot of time with different organizations and have seen where the vulnerabilities tend to be.

So, from a speculation perspective, what could have happened here?

Joe Garcia: Typically, what ends up happening is somebody is just trying to take a shortcut, and the API key gets hardcoded into code.

Maybe the intention was to move that API key eventually to a centralized solution where it could be programmatically retrieved, but it never happened.

That’s the problem with phased approaches—something gets put off until later, but then another priority comes up, and suddenly, what was meant to be temporary becomes permanent.

We don’t know exactly what happened here, but we can assume it was not secured properly.

Had it been secured and programmatically retrieved, the entire timeline of events would have never occurred.

David Puner: Do we know who the attacker is and what their motivation was?

Andy Thompson: The assumption from the federal government is that it was the Chinese government.

How and why? Public data doesn’t say.

And it’s not because of any technical indicators. The indicators of compromise that BeyondTrust has provided so far include a handful of IP addresses—four IPv4 and five IPv6—all pointing to U.S.-based infrastructure, specifically in New Jersey.

So, none of the technical evidence provides direct attribution to China. It’s more circumstantial evidence—the timing, what data was exfiltrated, and the modus operandi.

Their playbook is intelligence gathering, and that’s what this attack seems to align with.

David Puner: Do we know what kind of data was taken and what the intention might be?

Andy Thompson: The specifics haven’t been fully disclosed.

But what concerns me most is that the significance of the data is being downplayed.

They’re saying, “It’s okay, folks, it was unclassified data.”

That doesn’t matter. The amount of data, the sensitivity—especially when we start talking about things like sanctions—this is really damaging stuff.

Whether it’s classified or not is irrelevant.

Joe Garcia: Yeah, I think the terminology is really what we need to focus on in the reports that are coming out.

Anybody familiar with the government’s file classification system knows that “unclassified” can mean a lot of things.

At the lowest level, unclassified non-sensitive documents could be just everyday administrative stuff—like to-do lists or generic memos.

But then you also have unclassified sensitive files, and that’s what was exfiltrated here.

While it may not be classified, it’s still sensitive documentation.

It could include draft orders, early-stage policy documents—things that allow a nation-state to predict future U.S. government moves before they happen.

So, just because it’s unclassified doesn’t mean it’s not a big deal.

David Puner: So, Andy, just to cut right through it—what went wrong?

Andy Thompson: So many things, David.

From the application development perspective, zero-days were the root cause. These vulnerabilities, which for all intents and purposes were unavoidable, could have potentially been discovered through better software development lifecycle practices.

But that’s just one piece.

Another major issue was the API key. We don’t know exactly how the attackers got hold of it, but we do know that it was compromised, and that was a major problem.

Then there’s the broader issue of third-party access. A vendor’s infrastructure was exploited, and that opened the door to a government entity’s systems. That’s a problem.

And finally, the attack wasn’t detected soon enough. The timeline of events stretched too long before action was taken.

David Puner: At what point did BeyondTrust become part of this story?

Joe Garcia: I don’t even know that we can place blame anywhere just yet.

We still don’t know how the API key was obtained.

Was it accidentally put into source code and committed to a public repository? Was it written down in a document somewhere? We just don’t know.

But what we do know is that BeyondTrust’s SaaS backend—segregated for federal government customers—was accessed using that API key.

That means it should have been treated with a much higher level of scrutiny and security.

Hopefully, a key takeaway from this breach is that when a SaaS-based environment serves federal entities, it needs extra protections—period.

Andy Thompson: And I really think it’s important to go back to a point you made earlier, Joe.

Had proper API key management been in place, regardless of the zero-day vulnerabilities, this attack wouldn’t have happened.

Is that right?

Joe Garcia: That is correct.

Even if we assume some unavoidable security exception allowed the API key to be exposed—just having it expire with a time-to-live mechanism could have prevented all of this.

BeyondTrust actually provided guidance on this.

They advised the U.S. Treasury to implement IP whitelisting, aggressive event monitoring, and secure API key management.

That guidance addressed the initial attack vector. But there were still additional defenses that could have been put in place further down the line to stop the attack at later stages.

For example, had they implemented a zero-standing privileges (ZSP) approach, even if the attackers had gained access, there wouldn’t have been accounts with standing privileges for them to exploit.

If endpoint security had been properly configured, unauthorized access from unusual geolocations could have been blocked.

So, while securing the API key was step one, there were multiple opportunities to contain the damage before it escalated.

David Puner: In your conversations with customers and security leaders, what are they asking about this breach?

And what recommendations are you giving them to protect their organizations?

Joe Garcia: This is already coming up a lot.

With all the press and attention this breach is getting—especially since there’s speculation about ties to China—it’s top of mind for a lot of people.

The main advice I’m giving organizations is to immediately audit their third-party SaaS applications, APIs, and vendors.

This doesn’t have to be an external audit—just an internal check to assess where their biggest risks are.

Now is also the time to demand more security accountability from vendors.

Organizations should require vendors to submit to regular security assessments, provide a software bill of materials (SBOM), and prove that they’re securing sensitive assets.

At CyberArk, for example, we provide third-party notices that outline our SBOM. Every SaaS vendor should be able to do this.

This kind of proactive risk management strengthens security posture and reduces exposure to attacks like this one.

David Puner: In the CyberArk Labs blog analyzing this attack, there’s mention of these so-called “unwelcome holiday surprises”—breaches that seem to happen year after year during the holiday season.

Why do so many major attacks seem to take place during this time of year? Is it just a coincidence?

Did the timing of this attack in December impact detection and response efforts?

Andy Thompson: There could be some truth to that.

Threat actors often time their attacks for when security teams are understaffed or on holiday breaks, making response times slower.

That said, I don’t necessarily think the timing of this particular attack was intentional. It seemed more opportunistic than strategic.

But we have seen many cases where attackers strike during long weekends or holiday periods to maximize damage before detection.

Joe Garcia: Yeah, I was reading an article the other day about OpenAI’s models slowing down in December.

Apparently, because OpenAI’s models were trained on human-generated internet content, they “learned” that productivity declines in December.

Humans, in general, tend to slow down during this time—there are more vacations, fewer people actively working, and overall less responsiveness.

So attackers might take advantage of that.

Another factor is the widespread moratorium freezes that many organizations put in place at the end of the year.

These freezes prevent changes to systems and limit updates, meaning if an attack happens during that time, there’s less agility to respond.

So it’s not just that people are relaxed—it’s also that organizations are operating in a mode where making major security adjustments is much harder.

David Puner: Do we know when the initial attack actually occurred?

Andy Thompson: We know when they detected suspicious activity—early December.

But we don’t know when the initial compromise happened.

It’s very likely that attackers were inside the system much earlier than December 8.

David Puner: And as far as more details being disclosed, is there a point where we just won’t learn anything else?

Andy Thompson: That’s the unfortunate reality.

CISA, BeyondTrust, and other parties involved have largely closed the book on this.

We’re unlikely to get more details—especially when it comes to how exactly the API keys were acquired.

The investigation has moved on, and I doubt we’ll see any further technical disclosures.

David Puner: Now, you guys are hosting a webinar about this breach.

How can people watch it, and what are you going to be covering?

Andy Thompson: Yeah, Joe and I have been going deep into this breach, and we’ll be covering even more in the upcoming webinar.

The blog post we published gives a good overview, but we can only go so far in a written format.

In the webinar, we’ll be able to go more in-depth, including some areas we didn’t cover in this podcast.

Joe is also going to talk about the best practices that were either overlooked or not properly followed.

Joe Garcia: Right—BeyondTrust provided some recommendations, but I think there are still gaps.

At CyberArk, we take a more holistic approach to identity security, and I’ll be diving into the specific controls that could have prevented this breach entirely.

I’ll also be covering how organizations can proactively assess their own environments to make sure they’re not vulnerable to similar attacks.

David Puner: At the end of the day, this is all about prevention.

We’re not here to criticize BeyondTrust—we’re here to learn from this and help organizations strengthen their defenses.

Andy Thompson: Exactly.

This isn’t about pointing fingers. It’s about using this as a learning opportunity to improve security across the industry.

David Puner: Well, guys, thanks so much for joining me on the podcast today.

Andy, Joe—always great to have you both here. Looking forward to our next conversation.

Andy Thompson: Thanks, David.

Joe Garcia: Appreciate it!

David Puner: Thanks for listening to Trust Issues.

If you enjoyed this episode, check out our back catalog for more conversations with cybersecurity experts.

Make sure to follow us wherever you get your podcasts, and if you have questions or comments, drop us a line at [email protected].

See you next time.