How can independent product testing be made more sustainable for emerging countries?

asheje's picture
07/06/2015 | Jeffrey Asher

Independent product testing—whether it's conducted by CITE in emerging countries or Consumer Reports here in the US–can be costly and time-consuming. How can we reduce the time and cost of product evaluation to best serve consumers, especially consumers at the bottom of the pyramid? Are there creative ways to capitalize on the wide availability of user reviews via social media and other free internet information to supplement product testing for emerging countries?

Vote up!
Vote down!

Points: 2

You voted ‘up’

Comments

Gaurav's picture

perhaps organizations can partner with schools and colleges - this would allow the students to apply the engineering principles in a hands-on way, while learning about technology evaluation and contributing to society..

for dissemination purposes, the use of a mobile platform would make sense, given the proliferation of mobile phones in the developing world.

asheje's picture

Let me take the first part of your question here and then I will respond to the second part where Ms. McKown responded.

I agree that there would be significant educational value to bring consumer product testing to undergraduate and graduate courses.  As far as I know, there are no US colleges where you can get such a degree although CITE is developing individual courses at MIT to this effect.  My CITE work with MIT graduate and undergraduate students has proven that this will work.

But, here is the downside.  The number one objective for colleges is to graduate their students.  This means that you have a revolving door when it comes to students who perform these rigorous product evaluations.  They develop new test methods, create lab set ups, do the testing and they are gone.  Training new students is expensive and time consuming.  The very low staff turnover at Consumer Reports has been a big plus for developing a sustainable business model where timely, high quality test results for so many consumer products is published.    

Jeff

lmckown's picture

I like the idea of using a mobile platform not only for the dissemination, but also for the data collection itself. An information push and pull similar to what Yelp does.

Does anyone know of mobile apps that are good examples of crowdsourcing data while maintaining quality control? Especially interested in any used in developing country contexts... 

asheje's picture

Gaurav’s query above about the possible use of a mobile platform for test result dissemination is an interesting concept whose time has definitely come.  For the last 10 years, Consumer Reports (CR) has a for-pay on line service with over 3 million subscribers called, “ConsumerReports.org.”  Compared to their print offering, Consumer

Reports, this on line service provides far greater number of model evaluations with a publishing cycle that could never have been conceived of before the Internet. 

Recently, CR has created an app called, “Ratings,” that shows these product evaluations in brief.  Let’s say it is an app with great potential, but is still in development.

Thus, for dissemination mobile applications are the way to go … Now!

Ms. McKown’s response about the use of crowdsourcing data is another matter.  Personally, I get as much product information from Amazon User Reviews as I do from Consumer Reports (CR).  I view Amazon as one of the best crowdsourcing tools today and it is free. 

Unfortunately, recent published reports have indicated that there is a widespread “cottage industry” known to fabricate user reviews.

Thus, this leads to two important issues that would need to be resolved to make crowdsourcing for product evaluation useful:

  1. Verifying the integrity of a user review.
  2. Assuming you could verify a user review, does the person have something valid to say about product reliability and performance?

How would you resolve these issues?

ccardoso's picture

Interesting question, Jeff. 

It seems to me that time and costs associated with product evaluations are highly dependent on  their quality and purpose. The key is not necessarily to reduce cost and time, but to make them adequate to the type of assessment that is the most useful for people living in poverty. In this case, it is possible to create evaluations that are cheaper, faster, and possibly available for local organizations to conduct. As far as I can tell, this is one of the potential avenues that CITE might explore in the future. The bottom line for me is to make sure that safety is assured -- so as long as we can make this process safe, there is room to simplify/cut cost in product evaluaitons.

Regarding the online reviews issue, I believe they can be a powerful tool, especially if used and understood in the context of its limitations. I say this because I am not sure the end users (the people at the bottom of the pyramid) will be the ones to write the reviews. It seems more likely that NGOs or other local organizations, with more access to this kind of technlogy, will be the ones taking the lead in the reporting. As an indirect measure, their reviews can be informative, but should be accompanied by fieldwork and primary data colleciton to validate the results that appear online. If these safeguards are put in place online reviews can be very useful for CITE.

asheje's picture

Hi Cauam,

 

Thanks for your comments.  Below are my responses:

 

THINK LOCAL; FOCUS TESTING: To reduce product evaluation cost and time, you are correct that one needs to focus on the most essential issues (or performance attributes) of concern to the low income consumer.  Unfortunately, too many times I found at CR and in my CITE research that an attribute may appear at first glance straightforward, but is not. 

 

For instance with Solar Lanterns as an example, our Uganda market survey showed that 50% of the solar lanterns were able to charge cellphones. Surely, it should be easy to evaluate a lantern’s capability to charge a cell phone. But, in the lab we were surprised to find that their charging effectiveness varied widely.  Multiple samples of one model failed to charge at all.  Some charged quickly and others very slowly.

 

Just to test this one key feature, new test methods needed to be developed, special apparatus bought and the lanterns tested multiple times with different cell phones.  This rigor is especially required of the independent testing organization since the worst outcome would be to mislead these particularly vulnerable, poor consumers as a result of a testing error. 

 

To your other point, having testing and other test support functions done locally does make sense in potentially reducing costs.  It can also mean sustainable jobs and an infusion of technical knowledge. 

 

Beside NGO’s, local colleges could be a source for product test support, but they have the problem of being a revolving door for lab testing personnel.  Perhaps, the best answer may be standards labs like UL with local offices in developing countries.  This could work if they saw an advantage to broaden their business model to product performance testing.  Such a solution would result in having a sustainable, trained staff available for testing.

 

USER REVIEWS: I agree that NGO'S may have the best vantage point for gaining product market and use intelligence.  They can survey the poor for positive and negative issues in the use of a product.  And, they are more likely to have a web platform to disseminate key findings outside of their local area. NGO’s would also be more trustworthy for writing objective reviews.  (But, since they may also sell these products to the poor, their assessments may not be viewed as totally objective.)


As mentioned under “Think Local” (above), there remains the problem that these product assessments require paid, staff time.  You are correct that fieldwork and primary data collection would enhance the integrity of these reviews, but also would add substantially to the costs.  Markets change constantly and this would need to be an on-going effort. 

 

Jeff

Sir,I am working in an NGO named BAIF (INDIA). Our work is focused on rural livelihoods and we are planning a project on developing rural enterprise model on solar lamps and solar lighting system. Our approach is to link manufacturers who can supply parts of the lantern and they could be assembled and supplied by the community itself. So can you suggest a methodology to follow to quickly identify the quality of product from manufacturers so that we can shortlist companies for the project. We had already been part of million soul project, initiated by IIT Bombay and we have established hubs in rural areas of Maharashtra and Rajasthan. We are planning to initiate our project from these hubs. Your feedback would be valuable. 

Hi everyone,In case you missed it, Jeff wrote a great piece about this discussion that was recently posted on Devex. You can read it here: https://www.devex.com/news/the-robin-hood-model-a-sustainable-product-te.... Let us know what you think in the comments of this thread!

hpanday's picture

Hello Team CITE,

I am highly impressed by your initiative to evaluate products and formulate a framework for product testing. You are addressing a real problem here.

Based on my experience with field research in India, I had these problems while being in the field for pre-market testing of products.

1. Consumer's feeling of gratitude for free product sampling subconsciuosly replaces negative feedback to moderate response

2. End user's experience with the product is lost as there is significant gap between the first day of product interaction and the day of data collection

3. Consumer may feel intimidated or being judged by the surveyor and as a result, faking qualitative response

4. Cognitive overload of big questionnaire provoking unreliable response

5. Having no reliable means of cross verifying the provided information.

These problems require exploring tangible variables and mapping them with intangible consumer inputs to evaluate products (Especially the ones where innovation takes place at a different edge than the end user). I was wondering that how you guys are tackling these issues?

One possibility is to explore working with tech companies like www.wholemeaning.com that scrape the social web for content and, using nueral network-based "AI" software, categorize that content by emotion - aka, user review. -Joseph

One possibility is to explore working with tech companies like www.wholemeaning.com that scrape the social web for content and, using nueral network-based "AI" software, categorize that content by emotion - aka, user review. -Joseph

Add new comment

Facebook account Youtube account Twitter account