What place do DIY solutions have in comparative evaluations with ready-made products?

hbclake's picture
08/05/2015 | Brennan Lake

Despite persistent demand for products that satisfy basic human needs, in many last-mile communities market failures prevent the accessibility of such products.  One short-term solution is the use of "Do-It-Yourself" solutions that can be locally sourced, disseminated and implemented.  Simple technologies like bio-sand water filters, evaporative refrigerators and composting toilets are just a few examples.  What place do DIY solutions have in comparative evaluations with ready-made products?  How can development practitioners determine whether DIY products will adequately respond to the aspirations of bottom-of-the-pyramid consumers, or if they are simply stop-gap measures?

Vote up!
Vote down!

Points: 2

You voted ‘up’

Comments

asheje's picture

This is an excellent question and one that MIT faced a number of times in selecting models to test for the CITE program.

To start, with resources always a consideration, Consumer Reports (CR) generally does not test prototypes or DIY models with a limited production run.  There are many stories where CR tried to do the right thing by testing a particularly unique products early—like a computer or refrigerator that never reached the market—and got burned. 

DIY models have designs that tend to change quickly and can do so without the consumer knowing it.  This can lead to misleading test results for the consumer. 

This is particularly true for DIY devices manufactured in more than one location.  Supply chain factors or product improvements (or the result of reduction in manufacturing cost) can have a dramatic impact on performance, reliability and safety.  

Each comparatively tested model is a large cost and time investment to the testing organization.  To test a DIY model that can change, or worse disappear altogether from the marketplace, makes their selection less feasible.

Finally, if the product evaluation is based on a model that has changed—known or unknown to the product tester—credibility for the testing organization can be called into question.

Yet, irrespective of the CR’s real life concerns for products in the US marketplace, DIY products are a reality and play a critical function in emerging countries.  These consumers who buy a DIY model are particularly vulnerable if the product turns out to be a poor value or unsafe. 

Thus, my answer to Mr. Lake’s question is “Go for It!” 

To do this successfully, we need special test procedures in place to protect the consumer when DIY-type products are evaluated.  And, to do so should be viewed as a long term commitment by the testing organization. 

To start, we need to be selective in choosing a model where there is a greater likelihood for it to be successful in the marketplace.  This includes knowing that the manufacturer has a successful track record for product introduction and design.  Ensure that they will be in business for the long term.  The DIY device needs to have gone through the prototype stage where its design and materials have “stabilized” as it is introduced into the marketplace. 

In many situations, it may be expected that these test selection criteria for a DIY model will not be met. In those cases, it is suggested that model evaluations still be made, but published in a less rigorous fashion as a sidebar or blog, rather than in comparison to other models in a Ratings chart.

Assuming these model attributes are met, it is prudent that the test samples be chosen randomly from the manufacturer’s warehouse.  If the product is made at different locations, the selected test samples should come from each location.

Choosing more than one test sample, usually three, allows the tester to assess (non-scientifically) the model’s performance variability. 

Finally, the tester needs to do due diligence with a longer term engagement by going back to the marketplace to periodically select and evaluate new test samples.  While contact with the manufacturer may determine if changes to the DIY model have been made, the only way to be sure is to do repeated, independent testing.

With the right processes and procedures in place, comparative testing of DIY models can improve the well-being of consumers in emerging countries. If the product is exceptional, comparative testing can highlight this and accelerate its chances for success in the marketplace.  Comparative testing may also suggest ways to accelerate product improvements.

Jeff

eilten's picture

Dr. Asher's response shows clear thinking and an in-depth consideration of Brennan's question. It is exciting and informative for us at TEL to hear from such a well suited and unique voice in this conversation. Additionally, it is reassuring to know the issue of DIY solution evaluation has emerged in conversations at CITE already in their product selection discussions.

I agree completely with the requirement of a special test procedure and not a simple alteration of current systems and testing plans for DIY solutions. I think in the development of such a methodology it would be prudent to consider all the words of warning highlighted by Dr. Asher, including: swift re-design worries; cost and time investment; manufacturing location variability; and the reputation of the testing organization should issues arise.
 
One area, mentioned above, that I wanted to highlight further is that DIY solution standards will fluctuate massively with different manufacturing locations. Additional complication arise when these DIY solutions are more in the form of best practice instructions for complete construction on site, such as the Tippy Tap (bit.ly/TippyTapTEL). Developed for point-of-use construction, there is little-to-no control over the materials used and manufacturing standards when this is the case. Such solutions are becoming increasingly popular as oftentimes they utilize items found day to day and therefore have little to no initial cost to the user. This form of DIY solution compounds the outlined challenges of comparative evaluation even further.
 
Can comparative evaluation work in these situations? Should the evaluation focus directly on the final product itself or take one step back and evaluate the instruction documents provided? Can a best-practice approach to creating DIY solutions be formulated from a comparative evaluation framework?
 
Looking into this family of DIY solutions as a whole, I feel a different approach to evaluation is then required, perhaps the comparative model does not fit and a more impact oriented evaluation is needed. Specifically, does simply having the instructions and construction knowledge within the community result in better quality of life for those involved? Is there a best-practice approach to introduce DIY solutions to communities to increase the level of up-take and self-sufficiency? Can stakeholders be encouraged to propagate the knowledge within the scope of their projects and beyond? 
 
Unfortunately, shifting gear and working to measure the impact of such DIY solutions rather than comparing to similar products requires rigorous field testing and close working relationships with the designers and stakeholders. This brings us full circle to the issues of time and resource management highlighted by Dr. Asher above. As you can see we at TEL are extremely interested in this issue and are keenly watching this conversation develop here and within the evaluation community.
ohorizons's picture

A thought-provoking question, Brennan, and a very important one. I think your question of "How can development practitioners determine whether DIY products will adequately respond to the aspirations of bottom-of-the-pyramid consumers, or if they are simply stop-gap measures" highlights the importance of community involvement in any project. If you are doing a project where you're implementing BioSand Filters, but the community wants a candle filter, then you're lkely wasting your time. Without full buy-in and approval from a community, your project is doomed to fail, particularly in the long-term. This is why we see so many "failed' development projects. The reverse is also true, when an organization or company tries to push a ready-made consumer product, such as a solar lamp, on a community. 

The advantage of DIY solutions is that if a community is making them, they want that it and (likely) will use it. They've taken the time to learn how to make the solution, gather the materials, etc. I also think the merit of a DIY solution lies in its empowerment. You're doing it yourself, you're not buying it or waiting for a supply chain to reach you; you know how to fix the technology and you know how to get replacement parts. Unless a supply chain is well developed for a ready-made product, that is not the case. A solar lamp or water filter may work for a few years, but ultimately will need a new part, new battery, etc. 

 

I have been working on DIY solutions for the African rural poor for over 20 years and know all too well how difficult it can be to get acceptance!
My efforts led to the formation of a pv solar NGO which almost immediately decided that the DIY approach did not go with the massive funding they had managed to acquire. Hence they now offer imported solar devices to the easily-reached less-than-poor!
My attitude now is that the only way for those with $1 per day to have useful devices is to find someone in the community not only able to do DIY but undertake running repairs. The conditions in many poor rural areas is such that even modern devices seldom last long so repairs are often needed. whether DIY or not!
As one of your contributors wrote:
“The advantage of DIY solutions is that if a community is making them, they want that it and (likely) will use it. They've taken the time to learn how to make the solution, gather the materials, etc. I also think the merit of a DIY solution lies in its empowerment. You're doing it yourself, you're not buying it or waiting for a supply chain to reach you; you know how to fix the technology and you know how to get replacement parts”.
How true!
In the introduction to this topic mention was made of several devices that can be made by any community however poor. One item not mentioned is the extraordinary ability of the black soldier fly to convert organic waste into high value protein. This puzzles me since it is used in so many USA homesteads as well as in industry!
If anyone want to know more, or better help, my email is biodesigndiy@gmail.com

lmckown's picture

Great new article by our discussion leader Brennan Lake in Fast Company, drawing on this discussion!!

http://www.fastcoexist.com/3055888/can-diy-create-better-products-for-the-developing-world

Add new comment

Facebook account Youtube account Twitter account