Some Fire Chiefs have devised a great way of showing success in their careers and within their departments. By altering the reporting criteria for call data they can manipulate the data. That data compared to former data (history) will show change. The problem is that by changing the criteria we aren’t comparing apples to apples anymore.
For example…
The prodigal department would more than likely show an increase in fires up until the post “America is Burning” era of the 70′s and 80′s. This steady increase was due to the increase in population and growth of cities and counties (more people and more houses = more chance of fires). The culmination of the report “America is Burning” was basically that this is going to increase until departments create fire prevention education. Through fire prevention education, firefighters have been able to save thousands of lives over the past 30 years by preventing fires.
Once we hit the bottom on preventing fires the decrease in fires leveled off. This created a problem for Fire Chiefs. All of that hard work that started 30 years ago with the beginning of fire prevention had paid off and had reached its limits on decreasing fires. I don’t mean that we should stop doing it, it has just done all it can do and as long as we continue on with fire prevention we will continue to stay where we are. That is a good thing.
I know that some of you are probably saying that we can do more in preventing fires and I agree. However, it is outside of the realm of the fire prevention education we all know. Think earlier detection, non combustible products and building construction, among others.
So here we are with Fire Chiefs trying to show how good they are and how good their department and fire prevention programs are and the fires aren’t decreasing. What is a Fire Chief to do? The answer that many of them found is to increase the loss criteria for structure fires. Whereas for the past 100 years the typical kitchen fire that burns the stove and cabinets above was a “structure fire” when reporting total fires is not included in those statistics anymore. If you raise the loss to say above $5000 or $20,000 to deam it a structure fire we have in fact removed many fires from the data. This in turn shows a decrease in fires. This type of thing is not done overnight. A Fire Chief might make the criteria go up a couple thousand dollars each year before meeting the goal amount.
These statistics are a problem within the fire service. This is a false sense of protection. These statistics have been used all across the globe to decrease staffing, apparatus, and stations.
The altering of criteria runs a lot deeper than the example I gave above. We see it in all facets of our jobs. Anywhere we can make things look different than they are to benefit the people holding the data it can be seen.
Response times is another big example of using statistics to show what “they” want it to show. Fire Chiefs and other administrators have found that by recording the on scene time for EMS calls works out a lot better than utilizing the much more exact “patient contact” time. Who are we kidding. On EMS calls we mark on scene when pulling up to the address, however how long does it take to get the gear and stretcher and take the elevator to the 21st floor to the patient. In reality, we should be utilizing both times. However, if we are only using one then the “patient contact” time should actually be the “on scene” time.
What do you all think?
Numbers lie and liars use numbers. Unfortunately there is no other reliable way to show effectiveness of a variable, than to count something and apply rules to the counting. In a project I am working on for my department I asked for every stat I could think of for every company in question. That way I can look at all the raw data and look for trends. Since I’m going into the project without a preconception, it will be easier.
I agree about the “at patient time” but US EMS is based on the 6 minute BLS 9 minute ALS inverted care model. Add a new variable to that goal and we sound slow when our average dispatch to at patient time rises above 10 minutes.
I think we need new ideas to track effectiveness of response. Not sure what those are yet.
HM
Numbers lie and liars use numbers. Unfortunately there is no other reliable way to show effectiveness of a variable, than to count something and apply rules to the counting. In a project I am working on for my department I asked for every stat I could think of for every company in question. That way I can look at all the raw data and look for trends. Since I’m going into the project without a preconception, it will be easier.
I agree about the “at patient time” but US EMS is based on the 6 minute BLS 9 minute ALS inverted care model. Add a new variable to that goal and we sound slow when our average dispatch to at patient time rises above 10 minutes.
I think we need new ideas to track effectiveness of response. Not sure what those are yet.
HM
I’ve read that many ALS providers don’t even use the right system for measuring cardiac arrest survival rates. Apparently the correct formula is called the Utstein template where only patients who had a chance are counted and survivors must have been discharged from the hospital without signification brain damage. Some systems cheat by defining a save as patient arrived at the hospital with a pulse.
I live in the Seattle, Washington area. Apparently we have the top ALS program of big cities measured partly by cardiac arrest saves. I would love to know what if any crap Medic One pulls in their stats.
I’ve read that many ALS providers don’t even use the right system for measuring cardiac arrest survival rates. Apparently the correct formula is called the Utstein template where only patients who had a chance are counted and survivors must have been discharged from the hospital without signification brain damage. Some systems cheat by defining a save as patient arrived at the hospital with a pulse.
I live in the Seattle, Washington area. Apparently we have the top ALS program of big cities measured partly by cardiac arrest saves. I would love to know what if any crap Medic One pulls in their stats.
I applaud those who even think about these problems! I have been a firefighter for 30 years and wondered long ago where they even got the data they used in the “America Burning” Report.
As I have observed the implementation of the NFIRS System (v4 and now v5) I am amazed that we are able to ascertain anything from the data at a National Level. I have been involved on the Local Level for 22 years in capturing and validating NFIRS Data. The biggest problem is that there are so few of us that are even aware that we are all responsible for the data (specifically entering GOOD data for the reports we enter).
At the National Level, it is amazing that the enforcement of the “NFIRS Data Rules” is not a high priority – specifically, I speak of the recent requirement that “if a 100 Incident Type code is entered, all the Fire Data Elements are required”. That code was never intended to be used to enter data with, it was a “Header” and to be used for NFIRS4 Translation ONLY. However, now that Software Vendors have been allowed to pass those records in the NFIRS Export into the National Database, the Feds have weakened the “Standard”. But even worse than that, in my opinion, the initial inclusion of the “611 -Dispatched and Cancelled Enroute” incident type code is totally detrimental to being able to quantify “Types Of Incidents”! They (the Dispatch Center) sent us out with sirens blaring and red-lights flashing for a reason – but his code says that they did it just to show that they could… We were going to some specific TYPE of emergency, but when we use the 611 code we never know what that was!
Assuring that your department captures good data is difficult, mainly because we all want to be able to enforce what we (our agency, county, state, and we personally) think is important AND none of the NFIRS Program Vendors have any tools to help us. There is a tool out there that will allow you to “enforce your Local Business Logic on what is good data” – check it out at http://www.MyFireRules.com – it allows any “rule” to be written to validate one piece of data against another or against a standard. It can be modified to work against any SQL Database.
Keep fighting the good fight!
I applaud those who even think about these problems! I have been a firefighter for 30 years and wondered long ago where they even got the data they used in the “America Burning” Report.
As I have observed the implementation of the NFIRS System (v4 and now v5) I am amazed that we are able to ascertain anything from the data at a National Level. I have been involved on the Local Level for 22 years in capturing and validating NFIRS Data. The biggest problem is that there are so few of us that are even aware that we are all responsible for the data (specifically entering GOOD data for the reports we enter).
At the National Level, it is amazing that the enforcement of the “NFIRS Data Rules” is not a high priority – specifically, I speak of the recent requirement that “if a 100 Incident Type code is entered, all the Fire Data Elements are required”. That code was never intended to be used to enter data with, it was a “Header” and to be used for NFIRS4 Translation ONLY. However, now that Software Vendors have been allowed to pass those records in the NFIRS Export into the National Database, the Feds have weakened the “Standard”. But even worse than that, in my opinion, the initial inclusion of the “611 -Dispatched and Cancelled Enroute” incident type code is totally detrimental to being able to quantify “Types Of Incidents”! They (the Dispatch Center) sent us out with sirens blaring and red-lights flashing for a reason – but his code says that they did it just to show that they could… We were going to some specific TYPE of emergency, but when we use the 611 code we never know what that was!
Assuring that your department captures good data is difficult, mainly because we all want to be able to enforce what we (our agency, county, state, and we personally) think is important AND none of the NFIRS Program Vendors have any tools to help us. There is a tool out there that will allow you to “enforce your Local Business Logic on what is good data” – check it out at http://www.MyFireRules.com – it allows any “rule” to be written to validate one piece of data against another or against a standard. It can be modified to work against any SQL Database.
Keep fighting the good fight!