Call for papers now open for the December Secure Code Summit! Register Here
Shopping Cart

No products in the cart.

Dumpster Fires: 3 Things About IR I Learned by Being a Firefighter

This webcast was originally published on July 18, 2024.

In this video, the speaker discusses the similarities and differences between incident response (IR) and firefighting. They delve into the methodologies and challenges of both fields, highlighting the importance of preparation, patience, and accountability. The speaker also shares personal anecdotes and practical lessons learned from their experience in both domains.

  • The webinar discusses the similarities and differences between incident response (IR) and firefighting, emphasizing lessons that can be learned from each field.
  • The importance of patience and avoiding tunnel vision in both IR and firefighting is highlighted to ensure effective and safe responses.
  • The need for accountability and structured hierarchy in managing both IR and firefighting incidents is emphasized for efficient and safe operations.

Highlights

Full Video

Transcript

Catherine Ullman

We’re going to talk about some things that I learned about ir from being a firefighter. We’ll see some stuff about how they’re similar and how they’re very different and what we can learn from that.

This is our basic agenda. I’m going to do a little bit about what ir is, what firefighting is, do some comparisons. We’ll talk about some opportunities and some challenges that I’ve learned from each of those things.

sum it up and I’ll give some final thoughts. So this is a little about me. for anybody who’s ever seen me speak or has heard about me, there is nothing on this particular slide that will be particularly surprising.

I work for Ub, as was mentioned. I’m on staff with a bunch of conferences, I volunteer with a ton of things. I’ve spoken a bunch of places. The important thing on this slide is I love sloths.

And that is flash. My adopted sloth, he’s awesome. The slide that most people don’t know a lot about is, my firefighting background.

So I was a state fire instructor. I have like, 54 credits toward a degree which I didn’t really need. I was a certified fire investigator, for quite a while.

I was a fire instructor. I have the national certification, and I worked with the j fire program, which aimed to help young folks, because typically the folks who are setting fires when they’re older do not simply start setting fires when they’re older.

They actually start very young. And the very first time that people usually see fire is on the candle of a birthday cake. So think about that for a minute and think about how people see that and what, they are learning, at that very young age.

Moving on. for anyone who doesn’t really believe me, this is me in the center. I am carrying what is called a thermal camera. And that camera can differentiate hot and cooler spots in a, building.

what we’re looking at, in particular, the reason I had the camera in my hand, we, were looking at a fire that we were actually at. And, you’ll see why this matters in a little bit.

But that’s me. And then this is also me. for any of you who, folks who watch Chicago fire, maybe, I’m a fan. I can’t deny it. the fella on the right is, David Eisenberg, who’s actually an actor on the show.

The fella on the left is Steve Chikarotis, who not only plays, on the show periodically, but he also is a real Chicago firefighter.

he’s retired at this point. but this was pretty awesome that I got a chance to meet them. So. All right, so to the meat and potatoes here. What is incident response?

This is a really basic definition that I’m not going to read you because you can do that yourselves. but it’s really just very, it’s meant to be very much an overview here, right?

So it’s everything you need to analyze and contain an incident in some, some way. And, that’s why I’m keeping this very broad.

The goal, right, to make sure we know what is going on, stop attacks in their tracks, minimize whatever badness they’re doing. And then obviously, in the end, we want to learn from what happens so that we can prevent those kinds of things from happening in the future.

We have a number of phases of incident response. Ideally, we’re always, always preparing, because that way we’re not trying to get ready.

Right. I think I heard this earlier today, that if you’re always prepared, then you’re never really getting ready. We want to identify when something bad like this is happening.

We want to ultimately stop it in its tracks and then get rid of it, recover and learn from it. So what about firefighting? Well, I mean, to some folks, this may seem really obvious, right?

We want to stop the spread, and extinguish unwanted fires. and I specifically say unwanted, because if you have a bonfire, a regular bonfire going in your backyard, and that’s legal, we certainly don’t want to stop that.

But again, this is an umbrella, description. It’s meant to be very high level. and these are our goals. So we want to protect public safety, the, health and safety of the public and the buildings that are important to us, private property, and minimize any kind of significant disruption, for community activities.

And, I mean, when I say all of these things, keep in mind that I’m focused on the fire aspect of this, because we also do things like motor vehicle accidents. many fire departments do EMS calls, heart attacks, that kind of thing.

But I’m focused predominantly here on fire. and we have a similar preparedness cycle. Right. We always want to be prepared for the next call. When that comes in, we have some sort of emergency, we respond to that.

we’re going to recover, we’re going to do some kind of mitigation to help prepare us for the next time and or help folks get ready so that they are more prepared for the next time.

So when we break down in particular, in the fire service, mitigation is going to be things like making sure people have smoke detectors or sprinklers in place, because those are the things that are going to help if there is another fire.

Right. we’re also going to make sure that we have maintained our gear. that’s where preparedness comes in. when we respond, we’re going to respond safely.

A lot of accidents happen to firefighters going to or returning from calls because they’re in too much of a hurry. and they don’t always buckles or seatbelts.

So it’s the same kinds of things that you might expect, you do every day. Right. And then we’re gonna, when we recover, we’re going to clean up, we’re gonna put our hoses back in the truck, we’re gonna restore the property as much as possible in terms of, whatever we’ve done to it.

And then, of course, the folks that own it will have to make some kind of repairs. Cause that’s not really up to us. All right, so remember that house fire we talked about?

So I want to bring up, these particular images. The image on the left is, again, me and my fire gear. And what I am doing there is I am changing an air bottle.

And for those who are not super familiar with this concept, we wear what are called scbas, which are, if you look at the right hand picture, you can kind of see the bottle in this corner, of the firefighters back.

And that is part of what we call an air pack. And the bottles are only good for a certain period of time. And if you breathe quickly, if you’re out of breath, if you’re working hard, they last even less time.

So after a period of time, the bottles that, we typically use, are 45 minutes bottles, but that doesn’t mean you’ll get 45 minutes out of them.

So when a firefighter comes out of the. Out of being inside the fire and they want to go back in, we have to change the bottle. So that’s what I’m doing. The thing that I want you to pay attention to is I want you to pay attention to on the right hand side, there’s been some flame impingement, and that, the blackened part you see on the coat here, that’s all this flame, impingement.

There’s also, if you look very carefully, there’s been some flame impingement on the strap. The strap has a hose that goes through it. That hose is connected to what’s called a regulator, that’s what you breathe the air through.

Ultimately, that’s what connects to a face mask. and that’s how all of this works in a very, very small, way.

What I was doing here was just changing this air bottle. Just doing my job, not thinking much of it. I went to take the old bottle off, no problem. I put the new bottle in place, I turned the bottle on, and I got a base full of air because the hose had broken.

And though I was wearing all my gear and I was prepared, I was unhurt. But this could have gone very badly. So, just kind of put this in the back of your mind.

We’ll get into some of the details, about this particular fire. All right, so some terminology that is often confusing to folks in this industry.

When we say ics, we are talking about the incident command system. So, for those of you who work in the ICs side of security, that’s gonna be super confusing.

It drove me crazy when I first got involved in this field, because, every time I heard ics, this is what I thought of. And it’s, I love this quote. It’s an organized response to a problem.

It’s, a methodology that allows, for a small incident or a large incident. We’ll look at this in just a second.

This hierarchy that can be grown out as needed. This is what we’re talking about. You have an incident commander, the person who’s in charge of the whole incident.

Typically this is some kind of chief officer, and we have a bunch of different folks underneath them. the public information officer who talks to the people, like in the news or the public, a safety officer who’s monitoring to make sure nobody gets hurt.

And then we have these other sections. the liaison officer typically exists in larger incidents, but the four at the bottom are pretty common.

And, we’ll talk about those in a minute. Here is how this breaks down. So the IC defines the goals and the operating objectives. And if you’ve been doing ir for a while, some of this may start to sound a little familiar, and we’ll see how these compare in just a second.

But operationally, we have somebody who’s doing what’s called operations. So they’re dealing with the strategy and the tactics, right? What is actually going to be done?

Who’s going to take a hose to what floor, that kind of thing. Resilience. And command is looking at the bigger picture. If you have a larger incident where we need to bring in other people or other things, perhaps, some heavy equipment, then, or other personnel, then we have, if it’s going to be long range incident.

So think something really unpleasant like a hurricane, situation where folks could be on the ground for a long period of time, then we need people from planning and admin and finance to help with.

With the rest of these details. So now let’s compare these two a little bit. So the methodology you might have seen is really similar, right.

Even though we didn’t officially have something that says lessons learned in both categories, we really do have a lessons learned section. But for incident response, our preparations are really the same as firefighting mitigation.

And, because the preparations in IR and putting something like a smoke detector in place, if we’re preparing as an IR person, we’re preparing by making sure that we can see the logs.

We need to be able to see that we have access to all the systems. We need to have access to that. our detections are working the way we expect, that kind of thing.

with, incident response, we’re going to identify something’s gone horribly wrong. Well, in firefighting, obviously, we’re going to respond and do the same thing. So these methodologies are really, really similar, but I want to point out what I think are some interesting misconceptions.

So, in Ir, every event is an incident is something that some folks believe in. Now, it’s okay to treat every event as an incident so that you don’t get complacent.

and that’s important. But it is not true that every event really is an incident. We certainly don’t handle every incident the same way. We may have the same methodology, or path that we follow.

but depending on what we uncover, we may go in one direction or another. It really depends. In my case, at the university, we don’t define what an incident is.

We let legal define what an incident is because that has a lot to do with how we can proceed. So we can identify there’s a problem and we can talk to the folks in legal and say, hey, here’s what’s happening.

How do you want us to handle this? but we don’t necessarily decide ourselves. Use the verbiage incident, certainly. in ir, we definitely see some incidents are solved very quickly, and some are nothing.

and you don’t necessarily need to be some kind of rock star, to be in iR. If you’ve seen folks that are really good at ir, that’s awesome. But we need everybody on an IR team, not just the folks who appear to be rock stars.

And I’ll tell you right now, even the folks that appear to be rock stars are just humans that maybe have a lot of experience or sometimes we get lucky, we just happen to see something.

and attribution’s not always possible. I mean, sometimes it’s very, very difficult. Firefighting. We’ve got some similar kinds of misconceptions.

for those of you who are unaware, firefighters are not always paid. I’ve been a firefighter for 27 years, and I’ve never been paid a dime. I’ve been a volunteer the whole time.

And that is true for a good majority of the country. except in large cities, most firefighters are not paid.

So that’s a thing. And a lot of people don’t realize that. note, firefighters are not always big, tough dudes. Not a big tough dude last time I checked.

but that is, certainly what we see on television sometimes, right. we don’t fight fires in exactly the same way because it depends on the situation. If we have a little dumpster fire or, like a mulch fire, we’re not going to handle that the same way as we are a multi building fire, which I’m thinking of right now, because the city of Buffalo just had, a major, fire last night, which was very sad.

There were several brick buildings, in a historic portion of the city that burned down. so they didn’t attack those fires the way they might.

If it’s somewhere where they have to go in and make a life rescue, they did that one from the outside. Because these were abandoned buildings, we certainly don’t quickly extinguish all fires.

Last night, the fire I was talking about, a, great example of that. And again, not always a full time job. I’m a volunteer. I do it as I get calls and when I’m available.

So definitely some misconceptions on both sides, but there are also some major similarities. In both cases, we have to focus on what the immediate issue is first.

So whether it’s a, we have a situation with a machine that perhaps, has some sort of malware on it, or whether we’re talking about the fact that there’s flames coming out of a building, we have to think about what we’re going to do about that immediately and be concerned about the cause later.

as somebody who did fire investigation, you’re not dealing with figuring out origin and cause immediately. You’re dealing with the life safety issues and the, the safety issues of the buildings and the property and whatnot.

we’re going to triage. Triage is absolutely key in both cases. So whether we’re talking about a situation with a whole bunch of systems and we have to focus on a couple that are absolutely critical for a particular organization, or we’re talking about a situation where maybe we have multiple victims and we have to take a look at the life safety issues and make some determinations as to, who we need to focus on first because their life is at greatest risk.

they’re cyclical. We saw that with the different stages. Right. We were just looking at those. they often both require thinking outside the box. This is one of, the stories I really enjoy.

So in ir, you might have to use a different tool because you don’t have the tool that you normally would use. Or maybe you need to use a different kind of adapter because, the adapter you normally use broke or something.

We think outside the box all the time that way. Well, firefighting is the same thing. I have a good friend who literally had to go to Home Depot and buy a bunch of stuff and build something because he had to move a very large individual out of a very small room on a second or third floor, and they didn’t have the equipment that would help get this individual out.

So he punted and he went and bought the things and built something and was able to get the person out. So real significant similarities. And we often bring outside folks in.

Right. Whether it’s firefighting, we call in what’s called mutual aid or incident response. certainly the university has an IR partner. We have the capability of bringing them in if we have a problem.

And sometimes they’re inside teams. So some companies are small enough, that they only can use outside entities, and some are large enough that, like where I am, where I mean, effectively a team of one.

But. But the point is that, sometimes you have inside teams with firefighting. there are companies, for example, that are factories, and factories often have their own fire, suppression folks.

So they’ll call in the local fire department, but they also have their own people. All right, now come the learning opportunities and challenges.

So the number one thing that I think is really important, whether you’re in firefighting or ir, is patience. I like to call this take a beat because do you really have something serious happening?

Because it’s very easy when somebody screams wolf to think there is a wolf, in fact, at your door. but sometimes it’s a mistake and think about what that best course of action is because you might find that what you’re doing causes more problems than if you took that moment and ran in.

So I had an instructor used to say, slow is fast. And what he really meant was being methodical. You wind up being faster than just trying to race through everything.

So remember that fire that I was looking at, that thermal camera through? This is that fire. These are two different images, same fire. when we first roll up on the scene, what we’re seeing on the left hand side, you can see just a little tiny glimmer of fire in this corner down here.

And we see some, smoke coming out of the eaves with that camera. What I could see was that all of the heat and smoke were up in the eaves of this fire.

They were not blowing through the roof. Unfortunately, some of our team did not focus on looking at a thermal camera to see where the problem was. They just saw the flaming bits and went after that.

And so ultimately, they cut a hole in the roof, which is what you see on the right. Excuse me. And ultimately, we often do that sort of thing.

and then we see the fire venting. But realistically, if they had taken just an extra minute, they might have been able to attack this and gotten to it before it got to the point where it was on the right hand side.

Which isn’t to say that it’s wrong, per se, right? I don’t want anybody to think that there’s necessarily a wrong way, but it might have been more affected if they’d taken that beat.

They’d taken that moment, thought about what they were doing, looked at all the things before they started jumping into it. So, another example in the IR space.

I don’t know how many of you are familiar with Chegg, but Chegg is a company that has had many, many breaches. In fact, it was so bad that in 2023, I think, the feds came after them and said, you have a real problem and you need to fix your crap.

But the reason I bring them up from a personal perspective is that, I had a supervisor who absolutely lost his mind because we were notified that we had a bunch of accounts that were compromised through the Chegg breach.

Now, what I mean is, Chegg had a breach. We didn’t have a breach, but we have students. And one of the things Chegg does, Chegg rents textbooks.

What do you think the odds are of a student using the same credentials for their school as they do for where they get their textbooks? Cause in their minds, it’s kind of the same thing, right, at school.

So we had a bunch of creds that were the same, and that was not ideal, and we had a bunch of compromises, but we. We were not compromised. So if you don’t take a minute and realize, like, this is not our breach, this is someone else’s breach.

Yes, we have work to do. That can be a problem. Also consider life safety. Right. So, and we’ll see that in a minute. But what is the real risk and implement and implication of whatever it is you’re facing?

one of the advantages I have as a firefighter and in the space where I’m in, I tip of the hat to anyone who works in healthcare and does ir and any sort of security having to do with healthcare or banking, because realistically, if things go badly, people could die.

But I remind our folks every day, no matter what we do, no one will die because we do not have those kinds of resources. So if you are in a company that doesn’t have life safety, that kind of life safety resource, take that beat.

It’s important. So another one, tunnel vision, right? So hyper focused on whatever it is you’re running in to do that, you can have a problem.

This is similar to what I was experiencing changing that air bottle. I was complacent, and I was hyper focused on the job I was doing. I wasn’t thinking to myself, gee, I should be paying attention to the state of this gear and making sure that, not only am I safe, but the firefighter is about to go back inside that they’re safe, too.

So this is another example, where that same fire where tunnel vision could have been really dangerous. You see there’s three cones.

They’re at the bottom of the stairs there. And if you look to try to see what they’re protecting, it’s next to impossible to tell. There’s nothing to see. Except if you were to look really, really closely, there’s a downed electrical line that’s actually at where those, cones are.

This was probably not the best way to mark that, but if they just had gone flying in the door, this could have been Dudley.

Similarly, we don’t think of this often, but is the scene safe? Why are we being called in as incident responders? Is this an issue because, a halon system has gone off?

Is there asbestos in the space? Is this a situation where are we screaming ransomware when it’s just some kind of weird configuration?

So just because you think you’re headed for an incident and you’re going to do things a certain way. Be careful not to get tunnel vision. You really need to do what we call a 360 and make sure you have a sort of global view of whatever it is that you’re handling because, you don’t want to blindly hardware that you don’t need to do any kind of research on.

Maybe, what was sent out was just a phishing message. It’s not ransomware. So then we have the opportunity of accountability, right? You need to know where your folks are on the fire scene.

This is literally life or death. We, have tags that you’ll see in a second, but even with an incident that we’re doing as ir folks, that is really important.

It’s especially important for anyone who’s doing work with law enforcement. We, need to make sure that, there’s no interference, stepping on toes, be cognizant of these physical hazards.

So these are, these are examples of actual accountability. Tags on the left from firefighters. And these usually hang off of our coat, usually one in your coat, one on your helmet. And if you go inside, you leave the, one of those two goes to an accountability person at the door.

So they know you’re inside and they keep track of, you on the first floor, the second floor. What have you with us doing? Ir, maybe swipe card access, who went in and out of a building?

Because maybe this becomes some kind of an internal incident, right? Maybe, you have an insider threat you didn’t know. maybe you’re working with law enforcement. Maybe you need some sort of an access log.

So we have about five minutes left. So I’m going to summarize here and say the things I’ve learned. Have patience. Take that beat before proceeding into any incident.

Doesn’t matter whether it’s firefighting or incident response. It can be very, very critical, both in terms of being successful at your response and in some cases, life safety.

Do this 360 the problem. Make sure you have a full grasp of what’s going on, and you may not right away, and that’s okay. But make sure you understand as much as is known before you start tackling that problem.

Location, location, location. Make sure where your folks are. Make sure that, people aren’t just running in and doing stuff because they don’t know what else to do. In, the fire service, that can be deadly.

in ir, it can be, it could be deadly. If it’s a situation where they’re running into a server room that, maybe had a lot system go off.

and then, at some point, I’m probably going to do a talk on the incident command system and how this could be adapted to ir. But it’s not unlike what we would think of where you want sort of one person leading the show and, it branching out so you don’t have too many people reporting to one person, which can be very, very complicated.

So I to do a quick, shameless self promotion, this book I put out last summer called the active defender. for those of you who are really new to this field, this, I think can be very helpful.

It essentially, it teaches you how to put yourself in the offensive security mindset, which ultimately makes you a better defender.

I was absolutely blessed to have Jake Williams as my technical editor. it’s available, like all over the place. And with that, I’m going to wrap it up.

And I always love to end with this quote. I may not have gone where I intended to go, but I think I ended up exactly where I needed to be because I actually started my career in the music industry and took a sharp left and wound up in it.

But I think ultimately, as this says, I’m exactly where I needed to be.