Balancing the sustainability of these items with the benefits they seem to offer society, which range from convenience to sterility, has never been easy, and is even more difficult now. First, there’s the fact that they must be continually replenished: produced and shipped over and over again to meet unending consumers’ demand for them (a fact which, as the shortage of disposable masks early on in the pandemic demonstrated, is as inconvenient as it is environmentally taxing).

What’s garnered headlines is the tendency of disposables—often some form of plastic—to leak out of the waste stream and into oceans and waterways, where they damage ecosystems. And when they do end up in the trash, they burden municipalities tasked with disposing of them and contribute to growing landfill waste. Recycling them, when it’s even possible, is becoming a less viable option as end markets for recycled content shrivel.

Despite these criticisms, long before the pandemic, the case has been made again and again that the various benefits of throwaway products still outweigh their environmental and societal risks.

The invisible lever behind these claims is a type of study, called a life-cycle assessment, or LCA. It is a method of analysis that has evolved over the past 60 years to give corporate strategists and policy makers the ability to make sustainability decisions by measuring the environmental impact of various materials and products from “cradle to grave.”

Environmental impact, however, is notoriously difficult to quantify. The idea behind LCAs is to act as a shared measuring stick to drive industry or policy toward sustainability. Yet rather than generate consensus, often they sow the seeds of discord. Loose methodological standards have resulted in studies plagued with inconsistencies. It’s not uncommon for two LCAs claiming the same objective to yield vastly different results. The result is public confusion, fierce debate, and stalled progress on some of the most pressing environmental issues of our time, such as plastic pollution.

Yet the LCA is only becoming more important in the world of sustainability. Some of the world’s largest corporations, as well as international bodies like the United Nations and the European Union, increasingly look to it as a tool for high-level decision making. In light of this, some researchers argue the time has come for an open conversation regarding the widespread misapplications of LCA, particularly on the subject of reusable versus disposable, and how to prevent them. They say that only by overtly acknowledging its limitations can the methodology become a real tool for driving environmental progress.

Dueling LCAs

Alison Keane, president and CEO of the Flexible Packaging Association (FPA), a trade group that advocates for flexible single-use plastics (such as grocery bags and juice packs), relies heavily on LCAs in her work. She uses them to assess the sustainability of everything from baby food packaging to laundry pods.

“A lot of people will discount LCAs, particularly if you do them on behalf of your own industry like I’m doing, because you get to decide the inputs, and based on that, you’re going to get an output,” Keane says.

To allay such concerns, Keane says her organization always uses a third-party consultant, such as the U.S. Environmental Protection Agency or the Sustainable Packaging Coalition. “We always use their LCAs, so that we hopefully can combat that misperception that you can just put whatever you want in and then get whatever you want out,” she says. “It’s really not that easy.”

But many in the field of sustainability argue that it is.

“LCA is most of the time used as a tool to make your stuff look good,” says Clarissa Morawski, who runs the international not-for-profit association Reloop and uses LCAs to help companies and governments make sustainability decisions. She has learned to be wary of those coming from the corporate sector.

“Typically, when it’s given to us by a corporate interest we don’t take as much interest in it,” Morawski says. “You get the glass one, you get the metals one,” she says. “Then all of a sudden you’re trying to pick through it and see how they landed with such good numbers compared to this one.”

A glimpse into the history of LCA gives insight into its contentious use in industrial settings. From the 1960s onward, lack of consensus over an agreed-upon methodology delayed LCA’s widespread acceptance by the scientific community, but that didn’t stop it from appealing to corporations. Coca-Cola, for example, sponsored what is now considered the first LCA in 1969. This famous internally conducted study examined a question that still plagues consumer goods producers: What packaging materials are best for the environment? Unfortunately, the company never made the report public.

By the late 1990s and early 2000s, scientific journals, like the International Journal of Life Cycle Assessment, were created and a peer review process established. And since 1994, the ISO, a non-governmental organization that seeks to create international standards in various fields, has worked to establish a guiding framework. Despite this progress, however, in many applications, the methodology still seems to generate as much controversy as it does consensus, often painting a picture of sustainability that validates questionable practices in industry, rather than challenging them.

One of the most notorious examples of this is in the analysis of disposable versus reusable food containers. In 1994, a landmark study by Martin B. Hocking, a chemistry professor at the University of Victoria, drew the controversial conclusion that using and disposing of 1,006 Styrofoam cups was still less energy-intensive than reusing a ceramic mug 1,000 times, painting a grim picture of reuse for the general public. Later, a 2007 LCA from researchers at The Netherlands Organisation for Applied Scientific Research, commissioned by the Benelux Disposables Foundation, similarly argued that reusable ceramic mugs were “the most environmentally polluting system in seven out of … 10 categories.”

The two studies, says Morawski, are “a perfect example of an LCA looking at the worst possible option for reuse and making that the norm.” The results have since been widely critiqued. In 2014, researchers from Ohio State University sought to get a more accurate picture by performing a similar study. They altered important parameters that had previously been neglected, such as inaccurate cup size and outdated dishwasher efficiencies, and obtained a much more positive view of reuse.

Shortcomings of LCAs

There is much about the mechanics of LCAs that make them vulnerable to such variability. These range from misunderstandings about their scope to flaws in study design.

“When we’re modeling these scenarios, they are almost always specific to … a certain circumstance,” says Simon Hann, an LCA specialist for the sustainability consulting group Eunomia. “If a company came to us and asked, ‘Should we use this material [or] that material?’ We can do very quick benchmarking or screening LCAs that allow them to make quick and easy decisions around a fairly straightforward question.”

He warns that problems arise when third parties attempt to apply the results of an LCA performed in one context to another more broadly.

Morawski, who recently teamed up with the University of Utrecht in the Netherlands to conduct a survey of independent, peer-reviewed LCAs, says that while there are sophisticated techniques for measuring certain categories of environmental impact, others are far less developed. One of her main findings has been the extreme emphasis the methodology places on “climate impact.”

Because they look at the whole life cycle, LCAs take into account everything from extraction to transportation, to create a “pollution quotient” based on energy expenditure, which is then used to either advocate for certain materials or reject others. The problem, argues Morawski, is that “we’re actually not getting a full picture, when a life-cycle assessment only reports on climate impact. It’s only telling half of the story around circularity. It’s talking about the design piece, but it’s not talking about what’s the impact after use.”

This is why things like Tetra Paks (the packaging used in juice cartons) score very well on an LCA. Though they often constitute a cocktail of different materials (a fact that makes them almost impossible to recycle), because a main ingredient (paper) is made from trees, it is considered to have a “biogenic uptake” on the LCA, meaning its materials have the ability to account for negative carbon, which balances carbon expenditure occurring elsewhere in the process.

“The idea is, the tree grows, it sucks up the carbon and then it regrows again, so it’s like a continuous loop,” says Morawski. “But what’s clear is that the life-cycle assessment didn’t even account for the fact that, in truth, that Tetra Pak isn’t recyclable. It may be ending up as litter. God knows what. … When I found out that’s how the math worked, it blew my mind.”

Marine litter is another issue

Meanwhile, the effects of marine litter have been all but absent from LCAs. To accurately measure something like this, researchers would have to first identify the scale of the problem—“a nightmare in itself,” says Hann.

“Every plastic product has a potential pathway to the marine environment. And then there’s the chance of that happening. And in different places, it might be more likely. So, there’s a whole set of permutations about the scenario,” says Hann. But most perplexing is how researchers will calculate what it means to put a kilogram of plastic in the ocean. While researchers are now working to rectify this omission in the LCA, Hann says that “we’re really not there yet.”

An inability to quantify these subtler, yet still highly pervasive, impacts are what give LCAs an inherent bias: They tend to privilege the impacts of production (which, for example, materials like plastic score well on because they are lightweight and low-carbon to produce) over the impacts of disposal (a measure for which, being difficult or impossible to recycle, plastics score poorly).

Keane says she and her colleagues at the FPA account for poor recyclability in their tests by omitting it entirely. “From an LCA perspective, we try to be very conservative [on recycling], so we give ourselves a zero, even though we know some are actually making it through and getting recycled,” she says. “We just say, ‘You know what, it’s not robust enough in the United States at this point to get any credit at all.’” Even discounting recycling, she argues that the environmental benefits of single-use film packaging — it’s lightweight, low in production costs and takes up very little space in landfill — still outweigh those of other materials.

In a market that revolves around disposable, these features are why flexible packaging now makes up about 19% of the U.S. market, over more traditional materials like glass and aluminum cans, she says. If they’re all headed to the landfill, one-to-one, a plastic pouch of baby food is “less polluting” at that point than, say, a glass jar filled with that same baby food.

But Hann wonders if, for certain practices, trying to quantify the damage inflicted is even the right approach. An LCA, after all, is only as good as the questions it purports to answer.

“LCA can misdirect you to spending years and years working out what the problem is, which industry would love you to do,” he says. “Rather than spending a lot of time going, ‘Well, how bad is it?’ maybe let’s just go, ‘Well, we know enough to know it’s bad. Let’s work on not doing it.’”

Tbe stakes are getting higher

As time goes on, both Hann and Morawski are witnessing the stakes rise for life-cycle thinking in the modern era.

“Rather than just small, ‘What cup do I use for my coffee?’” Hann says, he and his colleagues increasingly find themselves commissioned by governmental bodies “trying to understand what material should [they] be subsidizing? Or promoting on a national or supranational level? And, obviously, as you can imagine, it’s a lot more difficult to have absolute answers when it comes to those things.”

Responding to these concerns, he published a report in late September with the organization Break Free From Plastic called “Plastics: Can Life Cycle Assessment Rise to the Challenge?” in which he details some of the most common mistakes associated with LCAs. Misinterpretation of study results is a big one.

“Off the top of my head, I don’t think I can even think of a really good example where an LCA has been published, and then the study has ended up in the news and it’s been characterized in an accurate way,” he says.

What does this all mean for the public, on the receiving end of all the mixed messaging, especially during a global pandemic where questions of safety and risk are at the top of a lot of people’s minds?

The report offers suggestions for commissioners, practitioners and readers of LCAs. For practitioners, he recommends exercising caution around what is placed in an LCA’s executive summary, the high-level review found at the beginning, from which journalists and laypeople often glean the study’s take-home message. In the 2018 Danish study, for example, he criticized researchers for emphasizing “some things that shouldn’t have been highlighted in the executive summary, knowing full well that they would be picked up [by reporters].”

For their part, funders of LCAs (such as corporations and governments) should set aside budgets for peer review and should be careful not to make claims beyond the study’s scope. And for readers of LCAs, he provides a list of things to be considered before jumping to conclusions.

What does this all mean for the public, on the receiving end of all the mixed messaging, especially during a global pandemic where questions of safety and risk are at the top of a lot of people’s minds?

At the end of the day, Hann says LCAs were never intended to be aimed at individuals: “Unless they know your exact behavior and where you live and what might happen to everything you do, this study is invariably irrelevant to your daily life.”

He warns not to let the endless barrage of contradictory results from competing studies obscure basic common sense.

“Forget about all that,” he says. “Because that is all about people arguing over the details and the methodologies. … Realistically, as an individual, if you’re using fewer resources and you’re throwing fewer things away, on the whole, then the aggregate of those series of acts is going to be good. So don’t get caught up in the minute details.”

Karine Vann is a Boston-based journalist who writes about food systems, waste and the circular economy.