Problem Framing to Mass Production: 8 Best Practices in DIY Product Evaluation
When a person lives on less than $2 a day — as some 2.7 billion people around the world do — there isn’t room for a product to fail. Investing in a product that fails undermines future innovation by reducing consumer confidence and depleting scarce resources. How can you design your product or venture using evaluation practices and principles from the very first prototype through mass production?
Experts from around the Massachusetts Institute of Technology gathered for a DIY evaluation panel at the annual Scaling Development Ventures conference to share their best practices and favorite on- and off-campus resources for product evaluation.
Amit Gandhi, Co-Founder of Sensen & MIT Doctoral Candidate in Mechanical Engineering
1. Spend time in the field to observe and understand how users interact with the product. Learning about usage patterns, some of which may be unexpected, helps reveal needs.
For example, after purchasing or receiving improved cookstoves, we’ve observed that people often practice “cookstove stacking,” continuing to use their old cookstoves alongside the new ones.
Check out D-Lab Scale-Ups User Research Framework to learn more!
2. Validate survey results with other data collection methods. Incorporating technologies like sensors can provide more accurate information.
For example, an evaluation of solar lanterns by the Comprehensive Initiative on Technology Evaluation (CITE) found significant over-reporting of solar lantern usage by interview respondents.
One resource we recommend for learning more about improved remote data collection is Portland State University’s SWEETLab.
Chintan Vaishnav, Academic Director of the Tata Center for Technology and Design
3. Pressure test “technology push” tendencies by meeting stakeholders and asking system-level questions.
For example, you may discover you need to shift focus from addressing a water leakage to designing a different kind of water valve; or change venues from lab-based to point-of-use soil testing.
4. Identify your point of intervention (often different from end beneficiary). Understand the value chain, and nurture a capable, engaged partner on the ground.
There are many valuable village-levels ideas to address health, water, and agricultural challenges where the end beneficiary is an individual, but the most appropriate partner to help bring that idea to life is a local institution.
Maria Yang, Associate Professor of Mechanical Engineering at MIT
5. Getting user feedback on a provisional design concept is a critical element of user-centered design practice. However, the exact form and context of these provisional designs can impact how users respond to the same design concept.
Learn more by checking out Jasmine Florentine’s student thesis, “User Feedback in Design for Emerging Markets: Methods and Influencing Factors.”
6. Tap into the thriving micro entrepreneurial culture of developing countries to evaluate products and their enabling environments. Several successful products have been identified, and share the common denominator that they are perceived to support income generation. In the words of one designer, ““Income is the best hook” to customers.
Jesse Austin-Breneman and Maria Yang explore this concept in “Design for Micro-Enterprise: An Approach to Product Design for Emerging Markets,” for the ASME International Design Engineering Conference.
Daniel Frey, Professor of Mechanical Engineering and Lead Researcher for CITE Suitability
7. Expose the product to challenging conditions and monitor the effect on the product’s functionality.
For example, CITE developed a three-tier testing regimen to rate solar lanterns based on their resistance to water. These tests involved complete submersion and exposure to heavy rain in different operation modes, and revealed several interesting benefits and drawbacks of certain products.
You can learn more about this testing by reading CITE’s solar lantern report.
8. Choose a broad range of products to evaluate including products the poor already use and some that might be much more effective even if they are currently outside the desired cost range
For example, in India, the CITE research team included high-cost reverse osmosis filters in our evaluation and discovered locally-assembled reverse osmosis filters that are half the price of branded models, and may be more accessible for low-income families if financing models are employed.
Learn more about this evaluation by reading CITE’s water filter report.