Who sets the standards?

By Carol Brzozowski

A common complaint among people who specify, purchase, and install erosion and sediment control devices is the difficulty of comparing the performance of different products and techniques. Although test data are available for many products—some from testing performed by independent laboratories and universities, some from the product manufacturers themselves—test procedures are not always consistent, and different test protocols make it hard to compare results. For this article, a number of people from various parts of the industry speak about their own testing efforts and how the issue stands industrywide.

Maximum Limits and Defining Failure

For two years, Thomas Carpenter, president of Carpenter Erosion Control in Ankeny, IA, and Joel Sprague, a senior engineer for TRI/Environmental in Austin, TX, have collaborated to create new procedures for testing the effectiveness of sediment retention devices. Carpenter and Sprague ran 30 to 40 tests of different volumes, quantities, and sediment loads through several different sediment retention devices and have written a paper describing their procedure.

The procedure involves a water/soil weighing and mixing device, a nonpermeable channeled area to disperse sediment-laden water evenly across the sediment retention structure, an area to install a sediment retention device, and the ability to collect and measure the sediments released by the sediment retention device.

"Our main goal is to start developing a test that is reasonable and compares all devices equally," says Carpenter. "In most testing, what we're looking for is the maximum limits a device can handle. From there, this test also helps define failure—what the process of failure is and what we need to improve so that those sediment retention devices work better."

Carpenter has sunk his own money into the project and relied on Sprague's expertise to develop what he hopes is quantifiable criteria for the performance of most best management practices (BMPs) in sediment retention. Carpenter points out that he and Sprague have followed tight scientific procedures and data collection but, because of funding limitations, have not had third-party verification on the data.

"When you go to the trade show, a lot of questions people have asked is that with NPDES [the National Pollutant Discharge Elimination System], what is adequate control? Every site is different," he says. "They use language like it has to be 'adequately controlled' or 'properly controlled' or something that is vague with no numbers behind it, so people don't know if 100 feet of silt fence per 10,000 square feet is OK, or is it 100 feet for two acres?

"I think there is a huge need for specifiers to know the effectiveness and what a control area is for a sediment retention device under some storm event," he continues. "All we have are marketing plans. We don't have any data that say under a particular storm this is how all six of these different retention devices work."

Sediment Versus Erosion Control

Geosynthetic-fabric-covered silt dike during testing

Test site for sediment retention devices

The effort is at the heart of a problem acknowledged by those in the industry: manufacturers, those in the field using the products, and people at laboratories and universities that test products for both erosion and sediment control. Many want to see uniform testing, but the question that looms large is whose testing will be universally accepted?

The problem is particularly vexing for sediment control products. "There is quite a bit of research on erosion control blankets, but there is minimal, if any, on how effective sediment retention products are," Carpenter says.

"I think we have a pretty good handle on what works and what doesn't work in the realm of erosion control and surface stabilization," agrees John McCullah, president of Salix Applied Earthcare in Redding, CA. "I think there needs to be quite a bit more effectiveness research in the use of what I call sediment barriers—those structures that temporarily reduce the amount of sedimentation."

Sprague agrees. "There is virtually no sediment control testing being done," he says. "There is one American Society for Testing and Materials (ASTM) standard for silt fence efficiency testing, but it's rarely done and it focuses on the fabric itself rather than the installed system. In terms of the effectiveness of the installed system, to the best of my knowledge, the testing I've been involved in with Tom, either directly or with Tom as the sponsor through the Environmental Technology Evaluation Center, is the only thing that's been done."

Carpenter says what prompted his report was what he believed to be false marketing claims about the effectiveness of certain sediment control devices.

"Neither Joel nor I agree on the testing procedure they use—there wasn't enough flow to the device to decide if it worked or not. Any device you put up there in such a low flow would have been 99% effective. [The product] was not compared to anything else. To me, those are marketing claims and voodoo science. If you're going to make a claim, do something that is more of a stress test and compare it to two or three retention devices and see how they work. But to just test one against no control and say it is 99% effective is, to me, false marketing."

Carpenter says he and Sprague have been working on the test for two years and he has spent his own money doing so "because I think it needs to be done for the environment. Theoretically, the Environmental Protection Agency should send somebody to do some major research, so Joel and I have been trying to put together this protocol," Carpenter says. "Part of the reason we did these initial tests was to get some data. We weren't trying to prove anything about one product or another. It was to take it to the Environmental Protection Agency and tell the EPA this is a beginning, a test that could tell us potentially some effectiveness of sediment retention devices and ask for grant money. If somebody got a grant, then they could use this testing protocol if everybody agreed to it, because this testing protocol addressed the problems of installation dependency and volume that other tests, like in the rainfall simulator, can't."

Roger Singleton is president of Silt-Saver in Conyers, GA, where most testing for soil erosion products is done by the state's Department of Transportation (DOT). A former residential developer, Singleton says he was driven into the erosion control business five years ago because he thought there should be a better method. He is often frustrated by the lack of receptiveness to new types of products.

"A lot of the products they test are not tested for their ability to perform but are tested as to whether the DOT wants to use them or not," Singleton contends. "In Georgia, we have a soil and water manual that promotes methods that have been tested by the DOT. They are generic methods that have been in the manual for 20 years. As new products enter the marketplace, the federal or state agencies are reluctant to promote proprietary products regardless of their ability to perform."

He cites an example of a new type of silt fence that was field-tested with no failures but turned down by the state because it didn't meet 20-year-old fabric specifications.

Singleton is also frustrated by communications; for example, the Georgia Soil and Water Conservation Commission recently informed him that the state agency cannot endorse the use of any individual proprietary product, regardless of the potential advantage of one product over another.

"So we have an erosion control industry that is promoting performance products and we have products that have the ability to clean up the streams, but by the power of the government bureaucracy, these products are left on the shelf, and they [government agencies] promote generic products that have neither been tested nor approved for the methods that they are used for," he says. "We've got products that will stop erosion from going in the streams, but the states have to recognize these products for their ability to perform and almost ignore the fact that they are proprietary because, if we put our money into a product, we want to get some return for it," he says.

Singleton would like to see state-to-state performance testing on products' ability to perform in the field regardless of what he considers outdated specifications. He notes that even if independent testing proves a product works better than generic products promoted in the state manual, counties enforce only what is in the manual. "A developer can't use these products because they are not in the manual," he says. "So the circle goes round and round. A number of us who have developed new and innovative methods are very frustrated with the system because there are no performance criteria," says Singleton. "As new products are brought into the system, they're thrown into the same basket with non-performing products and treated as equal. Then you get your bidding system that comes along and our innovative products are considered an equal to the non-performing products—they are a lot cheaper, so they always win the bid."

State and University Testing

Gilbert Layton says his state, Wisconsin, is on the leading edge when it comes to requiring testing of products. Layton is the chairperson of the Wisconsin Department of Transportation (WisDOT) Erosion Control Stormwater Committee.

"We feel if you don't test, you are asking for failure, and we've probably been on the forefront of requiring testing of these products," he says. "If there isn't a standard test protocol out there, then we do our own testing. All of our erosion mats have to be tested using ASTM test methods."

WisDOT requires the following product information for product approval:

• A representative product sample—approximately a 10 ft.² sample for mat and geotextile products
• Product specifications, literature, installation references, field performance data, and lab test data
• Names of any other state agencies that are testing the product, and notifications of completed tests and/or product approval
• A completed Product Preliminary Information Sheet provided by WisDOT Technology Advancement Unit

Products passing lab tests are not guaranteed a place on the WisDOT Product Acceptability Lists. WisDOT standards must be met.

The state has developed standards for erosion control mats, Class I, II, and III; tackifiers; soil stabilizers; inlet protection; type FF fabric; temporary ditch checks; in-stream sediment traps; and articulated concrete block revetment systems.

Many in the industry believe universities have become a major source for developing unbiased independent testing. Among those who conduct such testing are North Carolina State University and Texas A&M University .

McCullah believes the industry needs independent testing to validate products. "I think [using] independent researchers—institutions of higher learning—probably would be one of the best ways to do it," he says. "The manufacturers are willing to put the time, energy, and money into providing the testing, but a drawback is you are not always going to have consistent criteria used."

McCullah is an instructor at Shasta College, where students conducted testing on erosion control blankets, constructing a slope and trying 13 different types of blankets. "We found it didn't matter what kind of blanket we used. It could even be straw mulch," McCullah says. "We had a 90% reduction in erosion. We got more than 25 tons per acre delivered from the little plots from raindrop erosion without any type of cover during the first two or three rainstorms of the season."

That indicates that if one uses proper surface stabilization, such as mulching with blankets, straw, or some other method, "you're probably going to solve 90% of the problem," McCullah says. "The raindrop erosion is probably one of the largest processes that produces sediment and breaks up soil particles."

Richard McLaughlin, an associate professor and extension specialist at the Soil Science Department of North Carolina State University, says he understands the industry's frustrations over wanting to have products tested consistently. "In science, as opposed to engineering, we just need to know what the conditions were under which the testing was done, and then we judge whether or not the testing is appropriate and make comparisons," he says. "Of course, it is a lot easier if everybody does it the same way, but I may not have the same question that somebody else has about the performance of a product, and so maybe I need to do a different test than the standard test. I agree that it would be nice to be standardized for the purpose of selecting the best product."

Staked straw wattle during testing

The university is attempting to do that, McLaughlin points out, but he adds that there is very little testing of products being done overall. North Carolina State University has set up a test site where researchers can direct a given amount of flow into a device and add the right amount and type of sediment and perform the test over and over under controlled conditions so that systems or devices can be compared "in a reasonably scientific and statistically significant manner so that we can draw conclusions," McLaughlin says.

"We are independent. We don't have any funding from industry or other private sources. It is all public funding. We are fairly objective. Even if university people have private contracts, all of our work is public and is independently evaluated in publications by other scientists, so university testing is about as independent as you can get," he adds.

McLaughlin points out that other universities, such as Utah State and San Diego State, also perform testing under controlled conditions.

"Both of them have elaborate artificial rainfall systems inside a building where they can tilt a bed of soil and create whatever kind of slope they want and test erosion control systems for that," he says. "Colorado State and Texas A&M have very large sites that allow tremendous volumes of water to be run through channels. I think our university and Penn State are about the only ones doing testing on sediment traps and sediment basins; most everybody else is looking at either erosion control products or channel protection products. We're at the bottom of the hill. When everything is all said and done, all of the water runs into a sediment basin, and that's where you need to do your final try to get the sediment out before you release that water."

Procedures used by North Carolina State University include controlling the flow by establishing a source pond that allows researchers to open a valve, send water down through a pipe at whatever velocity or flow rate desired, and then add soil. Devices and systems are then tested to see how well they work. Systems are changed out and compared to previously tested systems.

"It allows us to generate a runoff event, rather than [the device] sitting at the bottom of the hill hoping it rains," McLaughlin says.

The university hasn't done much independent testing for manufacturers, but instead has focused its efforts on state-funded grant work. The state "recognized there were no standard tests to tell them how things were performing," he adds.

The university also does demonstration projects in which a product is taken out to a site and a demonstration is run so that DOT or Department of Environment officials can see how it works. "It's more for them to see what products are available than it is a scientific evaluation," McLaughlin points out.

He says products are always tested using the manufacturer's recommendation. "We are not going to test it if it is not functioning properly, so that is certainly an issue," he says. "I know there have been tests done on silt fences, and it certainly does demonstrate a world of difference between a properly installed silt fence and a typical installation procedure."

Jett McFalls is an associate transportation researcher with the Texas Transportation Institute's (TTI) Hydraulics, Sedimentation, and Erosion Control Laboratory at Texas A&M University. The lab tests a host of erosion control products, including spray-on products, such as mulches, as well as roll-type blankets.

"We test them on their ability to establish vegetation and reduce sediment loss on slopes and channels," McFalls explains. "We are testing flexible channel liners and slope protection products. The result of this testing is what drives the Texas Department of Transportation's approved-products list."

McFalls says the testing began in 1990 in a full-scale outdoor setting. "We did do some artificial rainfall simulation, but these were on 1,400-square-foot test plots out on our outdoor embankment and our channels were 85 feet long, so those were full-scale tests," he says. "In 2000, we reevaluated our program and came up with a new set of tests and a new protocol."

One of the factors TTI was trying to avoid was catastrophic weather-related events that ended some of the tests.

"There were two times in that 10-year period where Mother Nature dumped about eight inches of rain over one weekend, and that pretty much wiped us out for that year because our test cycles were annual," McFalls says. "We'd install in March and test until about Thanksgiving, and in June we'd get one of those bad storms. Actually, there were the remains of a tropical storm out in the Gulf of Mexico, and when it came through here, it wiped us out twice.

"We decided, if we were going to build a new facility, knowing what we know now, what we would do different." TTI evaluated all of the comments and criticisms it had received over its 10-year period of operation and built a facility that addressed them.

"We still do basically large-scale tests; they are not what you would call bench-scale tests," McFalls says. "For instance, our slope protection tests are 6 feet wide by 30 feet long, and our flume tests are done in a 30-foot-wide by 18-inch-wide flume. Those are the two places we test for slope protection and channel protection. At the same time, we are also looking into bench-scale testing, which seems to be what's on everybody's mind right now.

"I'd like to be able to take a small piece of product and for a small amount of money and in a really short time be able to tell you how this product's going to perform. But I don't feel we're quite there yet," he says. TTI staff have examined existing bench-scale tests and produced some in-house in an effort to meet the demand.

In reevaluating the program, TTI decided it wanted to use different soil types to achieve testing results that could be translated into various regions around the country.

McFalls points out that as an institution of higher learning, TTI is not tied into any special interests. "We don't promote any kind of products," he says. "We're impartial. I think one of the hardest things is trying to get everybody to determine what's important. If you got everybody to sit down in a room and start talking about it, it would be really tough, because everybody has their own idea of what they think is important and what makes erosion control material successful. I would love to get it standardized."

Standard Protocols and Variables

In Charlotte, NC, Woolpert LLP, an engineering/architectural firm with multiple service lines, writes erosion prevention and sediment control standard specifications for the South Carolina DOT and Greenville County. Woolpert works to incorporate test specifications for testing facilities to develop an approved-products list. The manufacturers whose products are not on the DOT list of approved products often have their own tests done in their own lab or send them to an independent laboratory, with the results sent to the South Carolina DOT, says J.P. Johns, P.E., an engineer for Woolpert. The state examines the test procedures that were used to see if they match up with what's required in the standard specification and either accept or deny that specific product for the approved-products list.

"We do not write product-specific specifications," Johns says. "We have created precise standard specifications to incorporate the best products that are currently available. We have done extensive literature reviews and have conducted one-on-one interviews with manufacturer representatives, creating specifications that are going to provide municipalities with the best products available so they don't [risk] having any failure, which could lead to violations and fines."

John agees that there doesn't seem to be a standard test protocol that is universally accepted. "Several laboratories across the country are running tests, but they are not all following one single test procedure. We have looked at these procedures and have tried to pinpoint certain variables each test should take into consideration to help with the problems," he says. Typical variables include soil type, type of vegetation, vegetation density, slope angle, slope length, flume angle, flume length, and the width of flumes used in the testing.

No matter what the product, he points out—from erosion control blankets to hydroseeding mixtures to a bonded fiber matrix—variables like soil type are very important in the testing. "One product might work great on a clay soil, but if you put the same product on very loose sand, it might not be able to bond to the sand at all and it may fail."

An agricultural engineer with a strong soils background, Johns explains, "I am very interested in the interaction with the specific soil type. Some tests don't even list the type of soil tested, and that's the most important factor!"

Johns points out that many major manufacturers have their own testing facilities or use independent testing labs, such as those available at colleges and universities. However, it's still difficult for those using the products to compare them because the testing is not consistent. One lab might test erosion control blankets that can stand up to a specific shear stress and velocity, but the question is whether that specific test is repeatable, Johns says. "From the engineer's side, do you believe they are using sound engineering judgment, or is this a one-time test that worked perfectly so they are going to publish the numbers?" he queries.

The Erosion Control Technology Council (ECTC) is a nonprofit group working to provide guidance on the uses and installation of erosion control products as well as test procedures and standards. ECTC has handled the challenge by assigning products to "types" with numerical ranges, with lower-numbered types including a host of products that can handle a certain amount of stress, and higher-numbered types meeting more stringent standards but encompassing fewer products.

"That's one way to eliminate having the problem of putting in the wrong blanket," Johns says, noting that the system applies to many erosion and sediment control products as well, but "blankets are one of the more visible products being tested right now in the industry," Johns says.

Johns points out that many sediment control products list the flow-through rate, and although that number is instructive, it doesn't indicate how effective the specific product is in trapping eroded sediment for specific soil types. The industry's perception for a need for testing, he believes, depends on who is asking for it. "If a manufacturer makes a good product, they want testing. If a manufacturer makes a sketchy product, I believe they don't want testing. When you talk with the big players in the game who have a good product, they want testing."

State DOTs want testing because ultimately they are the ones responsible for the projects, Johns adds. "In my professional opinion, we need testing because of the Phase II requirements. When you get on the Internet to search for erosion control products, you get thousands of hits, not knowing how well the product actually works. Five years ago, you probably would only get 10."

Singleton, whose Silt-Saver has been tested by the University of Georgia, says he'll show potential clients the result of independent testing and explain that by using his product, they can get better results at the end of the pipe—but some still question why they should use his product when they can "get by" with something less expensive.

"We don't have an answer to that," the frustrated Singleton says. He says the main issue for him is performance testing. "If it doesn't perform, take it out of the book," he insists. "Erosion control products should hold back the silt, flow the water, and do this without failure. It's just three very simple things; it's not rocket science. If it doesn't perform in the field, it shouldn't be used."

Installation and Performance

An important factor to consider besides performance, Carpenter insists, is installation. He and Sprague make the point in their paper that sediment control devices are installation-dependent.

McFalls agrees with Carpenter's assertion that installation is just as important as the product itself. "If I've learned anything testing these materials for 14 years, I would say that has to be the key thing," he says. "Sure, you've got to select the right material and put it in the right applications, but if it is not installed correctly, that's a major thing. A lot of times, that gets overlooked. We've seen instances where a soil retention blanket was not successful on a job site, and as we look more closely, we see it wasn't installed right. We test based on manufacturers' published installation guidelines."

An installation snafu can be something as simple as putting in the wrong number of staples or failing to achieve adequate contact between the material and the soil.

McCullah also acknowledges that proper installation is one of the most important criteria for success. "That's what IECA does," he says. "With our training courses, we try to teach people how to install these practices correctly. If you don't install them correctly, you're not going to have a good system that works."

Improper application is another problem, McCullah points out. "If you are using a silt fence for raindrop erosion, that's an improper application," he says. "Through our training courses at IECA, we're helping people develop ideas about when they should apply certain practices versus others. I don't necessarily believe manufacturers should be the ones leading people to believe what applications should be used because sometimes a manufacturer would like to have their product used for all applications. These are the kinds of research that should be done by independents, to test when certain types of products should be used for certain applications."

McCullah's personal pet peeve is witnessing products such as sediment barriers or grade control structures being placed in channels meant to convey water. "We need a lot more testing about the effectiveness and possible hazards of putting structures in channels designed to flow water," he says. "There has been a lot of research of lining channels with various turf reinforcement mats and erosion control blankets, but the idea of sticking structures into channels using straw rolls as sediment control hasn't really been tested. Some of the work being done with silt fences is really important. We need more of that kind of research."

Specifications Across State Lines

One of the biggest frustrations in the industry is the state-to-state variance on what products are acceptable and what testing is done to deem them acceptable.

"Many times, the approved list of products from state to state are holdovers from very old standards that were approved years ago, and there is a glitch in the system to upgrade this technology to get a state looking at the new products and technology," McCullah says. "That's another big challenge for IECA to get the word out to those state engineers and people with design authority for large state and federal highway projects to get them to look at some of the new products and technologies."

Johns agrees that state variances can be a problem. "Ultimately it comes down to not getting fined," he points out. "You want to be in compliance with the MS4 [municipal separate storm sewer system]. If they have an MS4 permit, a Phase I or Phase II community, they want to make sure they are in line."

Acknowledging state-to-state differences, McLaughlin says independent testing would be transparent and work for every state if a manufacturer came up with independently generated data that would be valid in any state.

McFalls says some states avoid reinventing the wheel by adopting standards from other states—such is the case with several states adopting the Texas DOT's approved-products list.

Johns says that the specifications for blankets that Woolpert is writing for South Carolina's DOT is attracting the attention of other states' DOTs. "Because we have been in touch with so many of the manufacturers and the specification is endorsed by the majority of the industry, they have already let other entities know that we have a great specification," Johns says. "I know that other states have already been in contact with South Carolina DOT about this spec. Hopefully, with the time and effort we put into this, we can see this starting to roll over into other states."

However, regional testing may be in order, he notes. "In agriculture, fertilizers, pesticides, and herbicides are tested all over the country because they don't behave the same way from one place to another. There are different soils, different topography, and different weather patterns. For erosion and sediment control products, there's probably going to be the need for doing at least some regional testing to demonstrate how different products work under different conditions.

"Some products don't work so well when they have to go through freeze/thaw conditions, and other products are fairly immune to those types of conditions," Johns continues. "Every state has its own specifications, and that is a real problem for manufacturers of erosion and sediment control products. Some require in-state testing and some don't; some require a certain test to be done and some don't. That's an issue I don't know how to get around."

Johns says the best that can be done to help governmental entities is to write explicit specifications as opposed to vague specifications. "If you read some specifications from the past or erosion control notes on design plans, they say use erosion control blankets 'as needed.' In this situation, the contractor typically chooses a single netted straw blanket that is the least expensive option but is most likely inadequate for the task. We definitely can't have that anymore. We need to say—from the design-engineer end—that on a specific slope, for a specific soil type, a specific type of erosion control blanket is needed," he says.

Layton, who is part of a new organization called the North American Stormwater and Erosion Control Association, says other states and Canadian provinces are using Wisconsin's Product Acceptability List (PAL) as a blueprint for their own use. He says one of the goals of the organization is to address the lack of standardized testing procedures and protocol from state to state.

In the end, the question still remains: Whose standards will prevail? "We've been at this for 14 years," Layton says, offering a timeline. "We started out taking the only things we could get our hands on, and that was manufacturers' test data they generally had done at Utah State or Colorado State. Then along came Texas Transportation Institute's Hydraulics, Sedimentation, and Erosion Control Laboratory, and they were testing in real-world conditions and testing in soil with a large enough facility that they could install the product as the manufacturer recommended and test it. We liked it, so we went with that.

"Then the industry, through ECTC, started working on some test methods, and one of them is the ASTM protocol. There are several labs around the country that are doing that."

In Wisconsin, interim acceptance of products can be gained through testing by the American Association of State Highway and Transportation Officials' National Transportation Product Evaluation Program's Erosion Control Products Panel. Interim acceptance is good for 18 months; at the end of the period, large-scale testing is required for continued approval.

Presently, labs approved for testing of all erosion mats are Colorado State University, San Diego State University, E-Lab, and the Texas Transportation Institute's Hydraulics, Sedimentation, and Erosion Control Laboratory.

"Our goal in Wisconsin is to see uniform testing as much as possible," Layton says. "We encourage either through ECTC or ASTM that standard test methods be developed, and we've continually through the years changed the testing requirements in our PAL to try to keep up with what's the national protocol of the moment."

As for the work of Carpenter and Sprague, Sprague plans to submit the test protocol to ASTM to begin the process of having it accepted as a methodology for testing sediment retention devices.

Sprague points out that designers and specifiers need testing that indicates how well the systems work. "The makers of the systems seem to be much more satisfied with quantitative evaluation that can be in the form of successful field experiences," he says. "We don't see anybody reporting on the unsuccessful ones."

Carol Brzozowski is a journalist in Coral Springs, FL.

EC - September October 2004