For most fuel "saving" devices and products, the main evidence offered is testimonials from apparently satisfied customers. These typically follow this kind of pattern:

"I used to get 25 mpg, but since adding Product X to my car I now get 30 - that's a 20% improvement!" - Bob Jones, California

The first and most obvious point is that some of these "testimonials" may be simply fabrications - total inventions by the company selling the product. It would after all be almost impossible to check up on these stories. This is especially likely to be the case with products sold through network marketing (MLM), such as many fuel additives, where there may be thousands of small bottom-level distributors who will pop up on discussion Forums and offer glowing, apparently independent, testimonials.

But often the testimonials are real - users genuinely believe they have seen an improvement in fuel consumption since using the product or device. The basic problem here is that fuel consumption is extremely sensitive to driving style, type of journey, even the weather:

Here are some real measurements from my car over a 6 month period. The gridlines are about 10% apart. It's obvious how big the "noise" is, yet people will fit one of these devices and confidently state after just a couple of refills that they have an "11.37% fuel economy improvement", or something like that. Even motoring journalists, who you might hope would know better, are guilty of this. (A cynic would wonder if this is connected to the advertising revenue their magazine receives from the device's makers.)

Averaging the fuel consumption over a few thousand miles gives better results, but does not eliminate the variability, and also means that other changes (such as the car running-in) may be the true cause for the fuel economy improvement. This also means that the 30-day money-back guarantee offered by some companies has long expired by the time it becomes obvious it doesn't work. And most people do not have a good record of the fuel consumption of their car over a long period of time to use as a "baseline".

Equally, looking at instantaneous (or very short-term average) values from a trip computer or Scan-Gauge can very easily lead to big errors, since instantaneous fuel consumption is very strongly affected by almost imperceptible headwinds, accelerations or gradients. (For example, to drive a typical car at 70 mph (115 kph) takes about 20 kW. A barely noticeable 1% uphill gradient increases this by 5 kW (25%), with a corresponding increase in fuel consumption.)

Often some users report big improvements while others find no improvement or even a deterioration in fuel consumption. This is not surprising, given the variability you always see. Funnily enough, the companies selling these devices only report the positive results!

It's even worse, though, because it's not a blind trial. In drug testing, neither the patient or the doctor is allowed to know if they're getting the real drug or a fake, because it affects the results. When the driver knows they have a fuel-saving device fitted, it's bound to affect driving style. (If you inject a hundred sick people with plain water, most of them will report an improvement in their symptoms. That's why we have double-blind trials, control groups and the like - scientific testing, in other words.) With the greatest of respect to people who have tried these things, uncontrolled testing under variable conditions does not constitute proof!

This is a particular problem when users report an improvement in the form, "I get an extra 30 miles out of one tank of fuel now". Here you have to consider not only the many factors affecting fuel consumption, but also the variability of the fuel gauge, and just how "brave" you are willing to be before stopping to refuel. If you have spent a significant amount of money on a fuel "saving" product then there will be a natural unconscious tendency to run the tank down a little bit lower before refuelling, to convince yourself you have made the right decision.

You may say, though, that none of this matters. Surely if there are so many positive testimomials, then the product must be effective? Unfortunately history says that this is not so; there are a large number of products, supported by very many positive testimonials, which we now know beyond any doubt do not give a significant improvement in fuel consumption. Here are a couple of examples, a quick search with Google will produce a large amount of testimonial evidence for them:

Ecotek CB-26P: the makers typically claimed 15% economy improvement, and many users reported up to 25% economy benefit. Yet the makers' own scientific testing, carried out in 2005, showed conclusively that the true benefit was only around 1%.

FuelMAX / Super FuelMAX: the makers claimed up to 27% economy improvement, and offered a selection of testimonials to back this up. But the EPA's testing proved the device had essentially zero benefit, and the company has since been fined millions of dollars and banned from selling the product.

And don't forget the highly topical BioPerformance, which has literally hundreds of people singing its praises yet was shut down in May 2006 on suspicion of fraud. Since then, the company has failed to provide any robust evidence of a worthwhile economy benefit, and was forced to pay nearly $8 million as compensation to deceived customers.

The US Environmental Protection Agency, as part of its advice on fuel-"saving" products, says this about testimonials: Many ads feature glowing testimonials by satisfied customers. Yet, few consumers have the ability or the equipment to test for precise changes in gas mileage after installing a gas-saving product. Many variables affect fuel consumption, including traffic, road and weather conditions, and the car's condition.

The bottom line is: testimonials are not a good way to judge whether a fuel "saving" product really does what it claims. Even personal experience is not necessarily a reliable guide, since it is so easy to be fooled by changes in weather, traffic, journey type, (subconscious) driving style, etc.

Sometimes the testimonials are from companies rather than individuals: "Bob's Trucking Co. found fuel economy improved by 7.36% when using our product!", etc. It is of course true that operators of large fleets are very concerned about fuel economy, but in general, fleet managers are not experts on either experimental design or statistical evaluation - and are therefore almost as likely as a private motorist to be fooled into seeing an improvement that does not really exist. So testimonials from companies are not proof of effectiveness, either.

So how can you distinguish between effective products and those that are useless? The "gold standard" for emissions and fuel consumption testing is the emissions drive cycle. This is a standardised profile of speed against time; all new vehicle models are driven over this cycle on a rolling road to provide legal proof of compliance with emissions limits. The European cycle covers about 5 miles and includes both "town" and "country" driving, so is fairly representative of what happens in the real world. The test is run under absolutely controlled conditions, with even the air temperature fixed, and so gives repeatable results. This test - and only this test - is accepted by governments all over the world as proof that a particular car has the emissions and fuel consumption that the maker claims.

When a "fuel saving device" presents results from an emissions drive cycle, then there may be something in it. Any other sort of test data should be taken with a large pinch of salt! Many independent test facilities are certified to carry out this testing, and the process is not difficult or time-consuming. As a fraction of the hundreds of thousands of pounds a year profit made by some makers of fuel "saving" devices, the tests are not even particularly expensive. When someone makes claims that are completely at odds with current scientific and engineering knowledge, surely it is up to them to prove that their claims are true, with proper scientific testing.

If you found this page helpful, you may like to support my work. If you think I have made a mistake, or am talking complete nonsense, please take a moment to read the Response to Critics before saying so in public (on discussion Forums and the like).