Question 4 (opens 4 Dec.) What are appropriate targets/data to monitor our progress in “e-agriculture”

21 posts / 0 new
Last post
e-Agriculture's picture
e-Agriculture
Offline
Italy
Question 4 (opens 4 Dec.) What are appropriate targets/data to monitor our progress in “e-agriculture”

Question 4 (opens Dec.) What are appropriate targets/data to monitor our progress in “e-agriculture”, to demonstrate that post-2015 goals (or other development goals) are being met? Who should be responsible for these?

[NOTE: As one example of ICT4D monitoring, see the "Measuring the Information Society" report from ITU. For more about the post-2015 development agenda click here.]

Michael Riggs's picture
Michael Riggs
Offline
Republic of Korea
Some thoughts to start this discussion

As an introduction to this question, consider that development organizations, governments and the private sector need to show that their investments are paying off. This includes investment in the ICT4Ag sector. Yet our ICT4Ag sector is relatively new still, and there is arguably much work yet to be done in defining indicators and data that will validate the positive results we believe investment in ICT have.

Consider:

ITU prepares the annual “Measuring the Information Society” report. While full of useful information on ICT penetration and markets, it does not provide urban-rural disaggregated data or indicators related to the impact of ICT on agriculture. (The report can be downloaded here.)

UNESCO has started collecting data on a set of indicators to measure the use of ICT in education in Latin America. Could something like this be a guide for agricultural scenarios?

LIRNEasia’s Teleuse@BOP studies seek to define demand for and use of ICT by people at the Bottom of the Pyramid (BOP). Does this contain the data we need? If so, should it be applied on a global basis?

I look forward to your thoughts and suggestions on this very important issue.

kiringai's picture
kiringai
Offline
Kenya
Measuring ICT4Ag

Thanks Michael for the introduction.

This is definitely a different area and calls for a policy dimension that interaction with the grassroots may not have provided much insights, but yes there is reason to focus more on the measurement or impact of ICT4Ag.

My posts have consistently mentioned weight as the only standard that can provide measurable linkage to agriculture. Weighing, particularly when it is digital provides the foundation on which the assessment of the contribution of ICT to agriculture can be done. By measuring the inputs to the agricultural chain, and comparing that with the productivity or output from the chain using digitized  data from the databases of the aggregators of production, then we have a sure way of measuring agricultural benefits.

In my humble opinion, it is from the digital databases that the other tools - mobiles, radio, cellular communication, apps etc can gain relevance. Critical in the data management is the organizational framework or infrastructure for hosting the digital or ICT platform on which all other devices and processes will gain the reason for their inclusion as contributors to ICT4Ag. I want to submit once more that creating standalone smallholder solutions without this basic organizational foundation is just but sinking sand. This IMHO may explain why so much effort in ICT4D impact has been minimal as no strategic thrust has been implemented, creating the many solutions that have been more donor/investor/sponsor driven than sector or impact focused.

As derived from Sloan's 3Ses, we easily aver that without structure, there cannot be systems and without systems there cannot be strategy. The introduction of ICT4D within the WSIS focus and the MDG focus, referenced in the ITU and UNICEF's documents, many development initiatives never factor in the institutional framework against which to implement measurability. It definitely is not possible to control what you cannot measure which explains why the direction of many development projects where development efforts take place never sustain. Arising from this argument therefore, it is easy to see why the outcome of measuring what is scattered yields as myriad results as there are interventions that stem from as many actors, which beats all sense in sustainable development.

Given this thinking, there is need to right the wrongs that project designers and implementers have made happen. With ICT bringing as many thinkers as it is able to bring as it is itself a unifying framework, let all professions join in, contribute their knowledge set, create communities of practice in a shared platform which deliverers measurable impact. My submission therefore is that we are now moving from data based technologies to knowledge based solutions that call for structural frameworks in the form of agricultural enterprises that are based on agricultural value chains so that the agribusinesses that emerge from these value chain engagements can support the processes that ICT brings, after which crafting strategies on measurable impacts is possible.

Continued...

kiringai's picture
kiringai
Offline
Kenya
...continued

To address the question at hand more clearly therefore, the targets of data to monitor from ICTs include the following:

  1. What investments have ICTs attracted to a community, a neighborhood, region, country?
  2. What new knowledge can be inferred from the way people do their work out of using the ICTs?
  3. What level of efficiency in time reduction in carrying out tasks can there be out of engaging with ICTs?
  4. How much has ICTs promoted the inclusion of marginalized communities - youths, women, the poor into the mainline economic mainstream?
  5. Have ICTs promoted the creation of new enterprises that enable more investment in new technologies? and,
  6. How much have ICTs influenced policy shifts at the macro level planning?

The mere introduction of MPesa in Kenya created vast readjustments in the way people related. Savings at the national level has been rising, the velocity of money movement has increased and general wellness can be avered to be associated with this innovation that uses the mobile for money transfer. BUT it influenced the Central Bank of Kenya to change and influenced banks to change. Out of the technology that has been championed by a marketing telecom company, many youths have gained entry into the mobile apps development and many other services have become possible including doing business selling air(time).

Going forward, the adoption of technologies needs what has been well argued in my previous post - create organizations that can invest in technology platforms. AND of course no one would appreciate this to be my post if I do not remember to integrate all this to the value chain...so it is necessary to ensure that all the potential opportunities that the various value chains present are the force behind which technology is adopted by all. Let those who have land resources use them with inputs from those who have them whether they are physical or non-tangible like knowledge, let those with money invest as chain actors to provide a service to those lower and let those that can consume also feed the producers with the payment and promptly so that the movement of financial resources fuels growth.

Kiringai

hmillerwise99's picture
hmillerwise99
Offline
United States of America
Struggling to understand

Kiringai, I appreciate your thoughtful response, but I am struggling to determine what your key message is regarding the question of "What are appropriate targets/data to monitor our progress in “e-agriculture”?"

When it comes to monitoring and evaluating the impact of our work in e-Ag, I think it is critical to be very clear about the indicators, and to ensure that we are looking at outcome indicators as well as proving causality. Many initiatives that I have seen focus on output indicators (e.g. number of people registered) rather than outcome indicators such as increases in productivty and income. I fully recognize that establishing causality is very difficult in this case, but that tends to be an issue of resources more than anything. If we could conduct RCTs on most e-Ag initiatives, we would have a more solid fact base on which to establish causality. However, RCTs are costly. Donor partners who are increasingly (and rightly) demanding better impact data need to acknowlege the cost implications involved.

Just because it's hard and costly doesn't mean we shouldn't do it, but we need our partners to walk that path with us.

kiringai's picture
kiringai
Offline
Kenya
Monitorable Targets

Hillary,

My approach is value chain driven. I seek to see business indicators that are linked to the use of a centralized information system that aggregates measurable parameters like the farmer demand for inputs..depending on what value chain. This should again be matched against the sales by the supply chain. With an organization buying from the suppliers and individuals making individualized orders, its possible to use the farmer organization to tell the individual picture of how much impact the ICTs have supported through the database queries.

Similarly all parameters associated with knowledge that can be captured through digital learning sessions can be captured and documented. Radio call ins can also be used to collect respoinses to the use of various agricultural initiaitves.

When farmers are paid through their organization, the wealth formation can easily be ascribed and with credit being possible through the payment supported procurements that derive their force from the sale of produce, productivity and wealth formation can be assessed and impact measured. Suffice to say that all begins with measurable farm outputs that comes from digital weighing then others are easy to measured and digitally controled.

The cost of the survey is not necessarily high, it is based on how the system is designed. We work with multinationals that pay their smallholder suppliers and have to respond to their Fair Trade and Green Forest Alliance engagements. They do that using systems we have developed and are able to provide since we measure everything from the weights of the produce delivered by farmers. They payment to farmers is again from one pot and can be ascribed to the use of our technology.

As a business model, the design of the trials complies with this thinking and can be as empirical as there are possible areas of focus that one seeks to address.

Alvarez Garrido Rafael's picture
Alvarez Garrido...
Offline
Spain
to measure

 
thank you, Mr Kiringai about your interesting thoughts.

I think that in agriculture we don´t have to do nothing different from what previously have been done in  other sectors with more experience implementing ICT into their processes , like industrial or services bussiness. During the last 40 years implementing ICT they have learnt how to  measure anything important related with their bussines, even from all perspectives (human resources, tecnically, financiallly, marketing,
merchandising chain etc)) and the key question for all their advances have been to find out and to use the appropiate indicators. An example could be  the Balanced Scorecard – BSC, created in 1992 by Kaplan and Norton, which is a method to measure the activities of a company in terms of its vision and strategy and provides managers with a global view of business performance. This new way of  measurement have change how we see our organizations and have created a lot of different ERPs solutions to implement this philosophy in each sector and bussines.

We only have to do benchmarketing with other sectors and this has been the focus in my work for the last 15 years once i realized that we were working in practical agriculture without any indicator or very few measurent devices. I agree with your phrase that " is not possible to control what you cannot measure".

If our focus is to improve agriculture bussines, then our first target has to be how technically we can improve the profits for the grower. We have to measure it and without succeed in this,  will fail.


Then we will need other problems to be solved, like gender, culture etc which i agree that should be included in each project perspective if we want to have succed. I agree with Mr Boyera that says that we are facing a holistic problem, and this is the reason  because to solution is so complex, because we are not only facing technical problems, but humans too (more complicated to be measured).

My perspective is that the first target is to have a common lenguage to measure agri-bussines results and processes. Lets imagine how was bussines in the world before we had a General Accounting Plan. At least any enterprise can measure its balance, but in agriculture we have the lack that we are not able to measure our key processes properly. This processes are the one that could give help growers to get profits if we able to help him  using ICT, and there is a big confusion how to use new technologies to help growers solve daily problems. In last 15 years have appeared a lot of new technologies and we have realized that using each one "alone" we don´t solve the problems and we are starting to get success integrating them. 

(continued)

Alvarez Garrido Rafael's picture
Alvarez Garrido...
Offline
Spain
 (continued from other

 (continued from other post)

In my company we have moved from developing an expert system from data from our patented sensor to measure plant stress to realize that we needed first a more simpler and holistic approach: create a list of indicators that should be used in any commercial crop where we include from very simple to complex ones from different origins:

  • from sensors to describe processes(plant, soil moisture, soil nutrients, weather, irrigation system)
  • from remote sensing
  • from manual data by a pen or a digital pen to describe any interesting plant performance (like physiological or economic results or using your weighting digital devices  ,photography..)
  • from weather forecast that learns from farms microclimate

We will calculate this indicators from the raw data from the database. We must identify the "cost" involved to have each indicator, so depending in the project and the add value of the crop we should recommed more or less indicators to be used. At the same time we have done several projects to create new indicators that integrates different data to simplify diagnosis (as an example to calculate daily plant photosynthesis work from different sensors etc)

I agree with you that "it is from the digital databases that the other tools - mobiles, radio, cellular communication, apps etc can gain relevance". What i have been doing this previous years is to understand that we needed that approach working with real growers problems. We have to create the "data base" of each experience, and teach the technitians how they can do it using different tech.  I agree with your sentence "what we are now moving from data based technologies to knowledge based solutions that call for structural frameworks". Thats what we are doing. The first step is create the database in each farm. Then we will start to learn. Even only with inference analisys, because we have all perspectives that affects our plants dily performance.

In our company w
e have moved our interviews with our customers from talking too much about a technology from asking them "whats your problem?. Our new approach:

  • fist point: identify the problems to be solved in each crop
  • Second point: To define a mid term plan of learning process and investment to identify which indicators they could use and how much it cost to get it. So depending of the investment we could get different targets and we must know this since the beggining of projects desing. If we work in a systematic way we will be able to explain them the final target and how they could get it.. if the invest money and time. it is similar to to process to implement an ERP in any enterprise.

So i think we have to push to understand how all new technologies could help us to solve real crops problems. What we have done in Spain is to promote different research projects but with a very practical perspective to test in real commercial conditions how we could test this new technologies to solve problems in the most important spanish crops (olive, citrus, tomato, viticulture, stone fruits). Thats the way we have learnt last 6 years and we are still doin so. We have learnt from experience and failure...

kiringai's picture
kiringai
Offline
Kenya
Thoughts resonance and challenges in African production models

Alvarez,

Your thoughts and mine have some resonance that can help mold a solution to the practical data collection end. Having the ability to isolate the various layers in the production to consumption continuum is critical to creating wealth and identifying clearly which parameters are critical when measured.

When dealing with smallholder farmers who do not have the wherewithal to own the technologies that can provide the data, the challenge can bring even a well designed assessment model down. The structured organizational model that farmers need has to be factored in as a measurable outcome of the cohesion of members. Their ability to create nodes, or collection centers, from where produce is collected is critical. If the organizational framework is created, such tools as the value chain, as thought through by Porter or the Balanced Scorecard that you make reference to, from Kaplan and Stratton, will make sense. If you do not have an organization or enterprise, then mapping the models from these researchers will be impossible.

The case for Africa, where we just want to own land, dividing even the very small uneconomical units, even when such ownership does not mean much to you, has been a serious challenge in implementing systems or processes to manage agriculture. There is serious need therefore to think through the models that may be difficult to implement the kind of testing systems that you may have the ability to do in Spain. It may mean that as we develop in Africa,  need emerges to aggregate the uneconomic land units to ones that can make economic sense and where processes can be introduced. To help out, we have to think around land and produce aggregation or aggregating inputs (in all their shades as you have well referenced them) as well as outputs (as demanded by the market) to gain economies of scale. With this, you then can understand why weighing is critical as it measures individual production (delivered to the aggregation point/Centre/sore), helps assess the outturn (production per unit of crop), and supports proportional payment for the proceeds from the market, if the farmers are organized into marketing groups/units/entities, etc.

In a nutshell the knowledge dissemination at the production chain will be different depending on the level of knowledge of the farmer; or their workers. It can therefore become more complex managing this scenario in places where land fragmentation is a challenge than it is in some of the agricultural environments that seem to flow from many of us in this forum.

I seem to therefore notice a very clear divide in our thinking on the technologies that can work for some areas like those twhere high end technologies as you seem to describe make sense . The measurement systems and processes will therefore be different and may call for more focused consideration depending on where they are domiciled.

Kiringai

Alvarez Garrido Rafael's picture
Alvarez Garrido...
Offline
Spain
My perspective is that we


My perspective is that we have to "understand" and learn how to use each of the new technologies that are available, even if they are used alone or integrated. Best if they are integrated. We will go faster if we have the chance to design the best practical experiences that will lead us to understand how to use them to solve problems in each kind of "agriculture", in less or more developed countries. I fell that we should have at least some critical indicators in any experience, to compare and to learn.

In Spain we have the luck that last 6 years we have been testing this technologies in field conditions with more than 16 companies and now we have "our" experience adapted to our conditions. In africa, we could use our experience of how to use the technologie but we will fail if we don´t consider the human perspective of all the things that have been said this days in different post (gender problems, training etc)  or what you have just commented about smallholder farmers and its organizations...

I think here in Africa or in less developed countries problems to be solved are more complex, as working with an spanish cooperative of small farmers... were the solution is more involved in a "human resources" problem than a technical one, but if we find leaders that want to "change" things, and we have a "field organization" that help him to promote it, then we could help them because we will let him to "measure" the results and profits of the "new strategie" that will convince rest of the growers...

best regards

sboyera's picture
sboyera
Offline
France
what AND how: open data

I think the question is key, but the challenge is how to have a measurement process scalable and trustable?
I don't want to hurt anybody, but i feel that the measurement process of the MDG as done in the past is close to a joke. Government are reporting about heir evolution of the different MDGS, but this is usually disconnected from the field and from what is happening on the ground.
Here again, i'm a great fan of bottom-up approach: I'm convinced that the definition of a new measurement process is possible through the aggregation of data from the different initiatives in the ground. This is where a revolution is possible: if each and every initiative would have a simple and easy way to share their data, ala open data, it would then be possible to use the collective intelligence to extract interesting trends and evolution. Unfortunately we are still far from this, mostly because all the popular tools, while often free and open source, are not making easy the release of anonymized data in open formats.
There is a new initiative just launched few weeks back called GODAN (Global Open Data for Agriculture and Nutrition godan.info) that looks promising in that regards.
I would be happy to hear what people would feel about a measurement process based open data & data sharing?

steph  

mmmayzelle's picture
mmmayzelle
Offline
United States of America
public data

Steph, once again thanks for a very thoughtful post.  I believe you are correct that publicly accessible data would enable a effective macro-scale approach to measurement of impact.  Of course, however, that approach is not without obstacles:

1- establishing practice. Perhaps the quickest way to encourage the practice of sharing data would be via large donors requiring it.  Over time such a convention could also come about via large projects setting the example, the "good reputation" such transparency may offer the project, and other public perception benefits.

2-business practices. If the project is a self-sustaining business, it may have valid concerns about making its data publicly available

3-project competition- unfortunately, projects tend to be competitive rather than collaborative, thus discouraging their willingness to make their data available to their "competitiors"

4-privacy concerns- how much information can you share about a user before you begin to invade their right to privacy and anonymity?  Especially in a bottom-up, user-generated system and/or a system that utilizes user information (ex: location, crops produced, season, consultation history, etc) to narrow the options given to a particular user when they connect to the system, the knowledge that data is publicly available may deter the user from taking advantage of what the system may have to offer.

5-situational factors - oftentimes keys to project success lie in how well the project understanding and accommodates the local culture (gender disparties, indigenous knowledge/practices, etc).  Obviously these are unique to each community and difficult to compare across project.  Using data alone, it may be difficult to account for such facets of the project and their role in the impact of the project.

6- consistency - even if every project shares its data, if there is no standardized way to measure impact, then those data will not be comparable.  

Thoughts?


sboyera's picture
sboyera
Offline
France
re: public data

Hi Megan,

Concerning your first point, you are totally right. However, enforcing the requirements in grant agreement can only be imho half of the strategy. It must be easy for practionner to do so. So i think there is a need for a first global steps:
*developing tools and extensiosn of existing tools that make the sharing of data easy for non-technical people
*developing an architecture with a (set of portals) where it is easy to publish and make available the data to the community.

Concerning all other points, I think there are critical too, but not specific to the domain. All initiatives in the open data world are facing these issues.
For instance, privacy/anonymization is not only important but legally required in most countries. However, there are now processes in place that allows e.g. government to publish all their data in such anonymized form. 
About business practices, there are different levels. You may well have data that are critical to your business models, while their anonymized version has no real value but are good for measurement.
About competition, you are right again, competition is happening, and this will not change. But it is essential for the domain to define the right boundaries of competitions, to ensure that while competition exists, it is not detrimental to the domain, or more speicfically to the development outcomes in the ground. This is where enforcing the need of sharing in grant agreement is imho the best way to define those boundaries.
About situational factors and consistency, this is again fairly common in the open data world: you may have lots of data that are very specific to the culture, communities and people you are working with, and in that case those data might not be helpful for people working in other regions. However, it is likely that those data are helpful for other organizations working within the same context but in different domains (health, etc.). 
About consistency, i'm not a big fan of standardization here, and in practice this is not the way e.g. government portals work. You publish data, and as far as your datasets are correctly documented, anyone can reuse them and mash them up with other sources. However, it would help if for a core set of informaiton, such standardization in vocabulary and semantics happens on the model of e.g. IATI for aid spending (http://www.aidtransparency.net/ ).

All in one, this discussion is bigger then just measurement. I think that there is a need to go for an holistic approach where publication of data is only one part of the equation. The other part is about increasing impact by reusing other org. data. It is about taking advantages of what other have done and not restart from the beginning. In my experience, you can convince more easily people to publish their data when they understand the value and ues the data of others. 
Moreover, it is important to note that because this open data revolution is already happening at government level and at international organization level, there are already tons of information available for free. Therefore, part of the ecossytem is already in place, and opportunities are already here.

cheers
steph

mmmayzelle's picture
mmmayzelle
Offline
United States of America
enabling data sharing

Steph, Thanks as always for a thought-provoking post.

The concept of making data sharing easy and common practice is a good one.  I'd be interested in hearing others' thoughts on the best approach to this/lessons from existing platforms.

*Implementor --would the ideal be a business?  think tank?  national organization? government entity?

*Scope-- would national systems be most appropriate, or would a regional or global system for data sharing have the most impact?  

*Motivation -- what would motivate projects to add their data to the platform?

* Design-- How could the platform be designed to ensure that all data types can be acommodated?  Would this data collection project also serve as a source of suggested indicators/ impact assessment tools?  

*Unity -- would copycat data sharing systems detract from the concept?  If so, how could this be avoided?


mmmayzelle's picture
mmmayzelle
Offline
United States of America
user satisfaction

Is customer satisfaction the ultimate mark of a successful initiative?

My own project uses outputs (ex: access frequency, bounce rate, number of topics visited, etc) as  "suggestions" of success--ergo if the product was not useful, no one would be using it!  But ultimately user feedback is the greatest measure of impact; there is nothing we value more than a user's comments on what they found useful and how the product could be improved.  

Surveys capture the cultural appropriateness of the project and its approach.  Especially given the short time frames of many projects, surveys also better capture what may not be yet visible in the numbers.  

I.e. there's been a drought since the project was implemented; harvest numbers haven't increased; families are farming together, and youth and women feel included in information access; female farmers state that their knowledge of GAPs has substantially increased and they feel confident that they can increase their yields.

Did the project meet its goals?  

These sorts of outcomes could only be captured by eliciting user feedback.

As with all approaches, survey has disadvantages: time-expensive, perhaps difficult to extract honest opinions.  However, its advantages may make it a worthwhile componenet of measuring progress. 


scorrea@grameenfoundation.org's picture
scorrea@grameen...
Offline
Colombia
Keep it simple

In Spanish we usually use this aphorism “Lo mejor es enemigo de lo bueno”. It can be translated as “the best is the enemy of the good” or “perfect is the enemy of the good” or just “keep it simple stupid!”

We know that we are dealing with a complex subject that can be affected by multiple factors that are constantly changing. So we have to decide whether to try to monitor something that can´t be completely monitored or to choose a reasonable amount of indicators that can give as a fair idea of what is happening and what may be the causes of that situation.

In that sense technology -especially mobile- can be a powerful partner or a big obstacle to simplicity. I will say that mobile technology has the ability to connect us directly to the field, process and report data faster and better, which translates in faster and better decision making. So why are we going to measure thousands of indicators when we can concentrate in the most relevant ones and then chose when to make a deeper research on a particular subject, group, etc.?

To keep this post simple and concrete I will recommend to anyone trying to monitor progress in “e-agriculture” the following things:

1.    Include simple socio-demographic indicators that can give you a good idea about the HH poverty level. There are many tools available such as the Multi Dimensional Poverty Index, NBI “Necesidades Básicas Insatisfechas”, Progress Out of Poverty Index (PPI), etc. Many governments have studies in which they conclude which are the most common characteristics of poor families. Choose indicators that reflect the basic needs that a HH should access specially the ones that your intervention can affect.

2.    Include a poverty measurement tool that is both simple, accurate and which results can be benchmarked with other regions, countries, etc. My suggestions are PPI and PAT.

3.    Ask farmers about their attitude towards their livelihood. Understand their struggles and pain points.

4.    Understand the farmer´s attitude towards the services different ag providers are delivering (credit, supplies, TA, etc).

5.    Concentrate your farm´s diagnostic tools on outcome indicators and on the best agricultural practices you think they need to adopt in order to achieve them. Understand your theory of change; measure if farmers are adopting best practices and if those best practices are producing the desired outcomes.  

6.    Benchmark your results with all possible data available from trusted institutions.

7.    Go to the field! Try to see if what your data is saying is reflected on what you see and hear from farmers.

whitgantt's picture
whitgantt
Offline
Colombia
Avoid common pitfalls in ICT4Ag measurement

As other participants have pointed out, technology and especially mobile devices open up enormous opportunity for capturing data at a low cost. While on its face, this good news, the ease with which mobile devices can capture numerous indicators and vast quantities of data has often been a detrament to good design in M&E in the ICT4Ag field. Interventions often capture far more data than they can effectively analzye and sometimes confuse capturing data with interpretting it in ways that improve results in the field. In that regard, the previous post on keeing it simple provides some easy to implement techniques to avoid the trap of measuring everything.

Another temptation ICT4Ag projects seem to fall into, as noted in other posts, is the tendency to over rely on output measures, and especially absolute numbers reached, and to then equate large scale and lower cost/interaction with impact. Because technology is a means for achieving low-cost scale, it's not surprising that many ICT4Ag initiatives quickly reach a scale that other Ag4D projects cannot. However, not enough effort has been put into measuring and communicating outcome related indicators or proxies associated with interactions generated from ICT.  

Lastly, ICT4Ag initiatives often jump directly to measuring impact before getting the operations right. While it's important to build in outcome measures from the start, understanding what's working and what's not and where users see the most value, should be a high priority during early stage piloting and scaling.

The previous post which suggests measuring user satisfaction, which can be measured via repeat usage, is a solid approach for capturing the value that users percieve. It is also one that can "fall out of the data" if designed well since most technology products capture each interaction and associate that interaction with a unique user, this is particularly true when users are paying for the service. Disaggregating usage by poverty level, gender and other characteristics, which can be captured via mobile registrations, can help practioners better design and target services. Analyzing usage by product or content area (e.g. cattle diseases vs. chilling hub services) can help programs zero in on where to focus new content and services and when to course correct. These types of insights should serve as the basis for deciding what to scale but unfortunately are often frequently overlooked in the rush to scale.

Similarly, capturing adoption of best practices that have proven ranges for productivity and/or quality increases can be another way to get at outcomes in a short period while awaiting phase II, more costly Randomized Control Trials and other impact assessment approaches.

Finally, in our own projects, the most effective indicators have been those that value chain players already measure. By monitoring changes in key indicators that are important to value chain players and that are measured regularly as part of doing business, for example number of boxes of bananas rejected for ripeness issues each week, we can assess how particular ICT4Ag efforts (for example sending SMS on when to harvest bananas) are affecting results and tie those results back to farmer returns (e.g. a decrease in number of boxes of rejected bananas is tied to an increase in weekly payments to a farmer, both of which are already measured by the commercializer). Linking these types of indicators back to a cost-benefit analysis that captures the cost of introducting technology into business processes can also make the business case for ICT4Ag initiatives by not only measuring impact but also demonstrating value that commercial players will be willing to pay for.


Kantiza's picture
Kantiza
Offline
Burundi
The common indicators of ICT4Ag are under construction

I think that the establishment of the same indicators is the beginning of a common language for the building of digital databases in order to monitor the impacts of ICT in agriculture and the forum alike is the best way for setting up the common understanding of those indicators nevertheless it has been proved that each sector of the Millennium Development Goals has its specific indicators. For my understanding, the  increase of productivity and income for small farmers remain the main factors to be monitored after the implementation of mobile applications in agriculture nevertheless the mobile applications are on horizontal line towards many sectors of development where the agriculture have its specific indicators and the business have to be measured with others indicators as it is shown by the indicators agreed by the World Bank: http://data.worldbank.org/indicator and of course the best farmer is not necessary the best trader.
Indeed, I think that the impact of mobile applications in agriculture is already a based evidence, as the mobile and interactive communication is being more and more the new way of living even for the poorest of the world meanwhile the exponential progress of mobile applications has no common measure with the low progress of agriculture but many relevant studies have shown that there is a causality between the penetration of mobile applications in the populations with the income driven by farmers in the agriculture and livestock: http://econ.duke.edu/uploads/media_items/houghton-daniel.original.pdf
Also, it is worth mentioning that such subject has been talked in the former forums and I remember that I wrote that the mobile applications in agriculture should not be the silver bullets for small farmers and that the manipulation could happened anytime in the canal of communication like mobile applications reason why we must arrive on the ground to remonitor what the remote sensors and others reports by ICT applications have evaluated remotely and I posted that“The right information is not shared among farmers and traders”: http://www.e-agriculture.org/.../question-1-market-information-users-mobile-technology
Besides, I do believe that the construction and the exploitation of digital databases related to the achievement of ICT4Ag are costly and are not in emergency nowadays for smallholders who need the minimum for surviving, by the way, I recall my  post named “The robust data collection is needed to boost farmers not dbases” available on the link below:
http://www.e-agriculture.org/forumtopics/question-1-icts-collecting-agriculturalsocio-
economic-or-me-data-open-11-june
Prof Antoine Kantiza,-         

Juan Forero's picture
Juan Forero
Offline
Colombia
PPI and agricultural sector

Dear all,
I just wanted to share a few ressources that could be interesting for the discussions and for further conversations.

Reports and Case Studies on the PPI: In this site http://www.progressoutofpoverty.org/case-studies-reports you can find all our case studies in order of construction. There is a short description of each one so you can choose which one you find interesting. I recommend the "Gratia Plena Social Action Center and the PPI®" case study, which is one of the firsts.
 
Please find below some extracts of a report we made on the use of the PPI by organizations. As you may see most of them are in a very early stage. I am including some that are working in the agricultural sector (you will find renowned players such as COSA, SFL and CIAT):

a. ASOCATI and Fundauniban
ASOCATI and Fundauniban are two organizations that have been using the PPI as part of the Community Knowledge Worker’s pilots that Grameen Foundation is conducting in Colombia. Fundauniban began using the PPI in August 2012, even though they made a small pilot in July 2011; ASOCATI began using the PPI in June 2012. Because both organizations are part of the CKW pilots the purpose of using the PPI is the same: to first understand the poverty profile of its associates or beneficiaries and to ensure that the project is focusing on the poor population. The second objective was to build a sample that could serve as a baseline for tracking the progress of the communities over time.
The PPI was administered by Community Knowledge Workers (CKWs) who use smart phones to capture responses from beneficiaries. Alongside the PPI, other types of indicators were entered related to socio-demographic, agronomic, financial, and technology access indicators. The idea is to analyze PPI data with the data from the other indicators, characterize different groups within the baseline group and to track the relationships between the behaviors of these different indicators, including food security. In the future they want to improve the focus on the beneficiaries and identify the possible causes of the improvement or worsening of their initial economic situations with the help of more in depth qualitative studies.
ASOCATI has collected 500 surveys and Fundauniban has collected 411 surveys. The collection methodology used by both is on a census basis of all small holder farmers (SHF). Currently Uniban has five CKWs in the field and ASOCATI has five CKWs and seven technicians. All the CKWs from both organizations have been trained in PPI use, and the management is familiarized with interpreting PPI data.

b. Centro Internacional de Agricultura Tropical (CIAT)
CIAT was hired by CRS to build a baseline for an impact study that would evaluate the results of a project in the border between Colombia and Ecuador called “The Borderlands Coffee Project.” It is aimed to help 3,200 smallholder farmers in conflict-affected communities to expand high-value market opportunities and reduce their vulnerability to hunger and environmental degradation. CRS and its local partners will be working with 1,600 smallholder farmers in the highlands of Nariño in Colombia, and 1,600 family farmers in the Amazon provinces of Orellana and Sucumbíos in Ecuador. The project’s objective is to help farmers increase coffee productivity and quality, as well as their income. In addition, it works to expand non-coffee livelihood alternatives and reduce vulnerability to hunger and assist with adapting to climate change.
CIAT built a survey[1] comprised of more than one thousand questions that was able to capture data related to the farmers’ socioeconomic characteristics, the farm’s characteristics, the different forms of farm ownership, production, commercialization, associability and general agriculture practices. They also captured farm geo-positions and applied two other tools: the Household Dietary Diversity Score (HDDS) and the Months of Adequate Household Food Provisioning (MAHFP) for measuring household food access, both by FANTA[2]. (to be continued)



Juan Forero's picture
Juan Forero
Offline
Colombia
(continued) The main

(continued)
The main objective for building the baseline was to test if the control and treatment groups were homogeneous in order to check at the end of the project if the differences between them are due to the intervention. Another objective was to have an “initial photograph” of the current status of the farmers in the municipalities involved. The survey provided information on the socio-economic situation of producers, their connectivity and access to information, property characterization, production and coffee marketing, and access to capital and division of labor (gender component), among others.
CIAT designed a stratified sample that was taken from the population of beneficiaries (1,600 for Colombia and 1,600 for Ecuador) and made a geographical dispersion map based on the number of small holder coffee farmers in the project’s municipalities for selecting the control group. In total they conducted 510 surveys in Colombia (228 for the treatment group and 282 for the control group) and 519 in Ecuador (235 for the treatment group and 284 for the control group). They began collecting the PPI in April, 2012 and spent one and a half months completing the process, using 12 surveyors in total for Colombia and 23 for Ecuador. The agronomic part of the survey was applied at the farms and the socio-economic part was applied at the beneficiaries’ households.
c. Sustainable Food Labs (SFL)
The main objective of this project is to gain a better understanding of the realities faced by small-scale sugar cane farmers. To accomplish this, SFL is collecting farm level data from certified fair trade, organic smallholder farmers in Paraguay.
This project is designed to accomplish two goals:
1.     Provide greater insight on the livelihoods and challenges of cane farmers; and
2.     Test the concept of a lightweight, cost effective set of core metrics for smallholders.
An initial baseline survey of 45 farmers in 3 organic, fair trade certified cooperatives was completed in April 2012 by SFL and local experts in collaboration with Fair Trade International’s Paraguay staff.
Additional surveys have been being completed since early 2013 with the support of the Ford Foundation, who has also founded the new PPI for Paraguay released in December 2012. These surveys have reached a larger number of farmers and include interviews with hired cane workers to understand the livelihood profile of this group.
In March 2013 SFL started using the PPI in Paraguay, where they work with sugarcane farmers, in order to complement their pre-existing performance measurement work. The aim of implementing the PPI with their larger original survey is to arrive at a straightforward assessment of their impact on the livelihood statuses of small-scale producers. The questions they aim to answer with their original survey and PPI are: “Who are you reaching?” “Are you reaching the poor with this value chain?” “Is poverty decreasing overtime?”
An essential part of SFL’s measurement of livelihood statuses is household income; however, this is very difficult to calculate quickly. In this approach assets are the best measurement of income and income potential; but, said assets must be assigned a value related to their given context which can be hard to determine and very time consuming. SFL thought that PPI could be a possible shortcut to circumvent this costly process of household income analysis and easily determine if value chains were reaching the poor and if producers are less poor over time.
The surveys were conducted in the field by technical assistants from six producer organizations and administered to a random sample of sugarcane farmers, all of whom were fair trade certified cooperative members. These technical assistants also gave tech assistance to farmers on different visits.
As of now, SFL has collected 300 PPI surveys in Paraguay; this data is just now coming in and getting ready to be reported. Besides SFL, other organizations such as Unilever, Sab Miller, and cegenta, are all in the thought stage of exploring PPI use; Root Capital has actually tested it.

Juan Forero's picture
Juan Forero
Offline
Colombia
(continued) d. The Committee

(continued)

d. The Committee on Sustainability Assessment (COSA™)
COSA works with cacao farming in Nicaragua in IFC funded projects aimed at increasing acreages planted with cacao. They have been working with Lutheran World Relief and Catholic Health Services who have likewise been using the PPI in Central America. The PPI is also being used in an evaluation of a technical assistance certification program in Veracruz.
COSAS’s data collection has been taking place in rural communities with producers of cacao and coffee in Central America (Honduras, Nicaragua, El Salvador, and Veracruz, Mexico) and Colombia. Usually the interview takes place at home or on the producer’s farm. COSAS’s surveyors talk to the farmers on their farms or at their homes and at the end of the interview the PPI is given as a separate interview segment. Depending on who the client is and what they are looking for, the entire survey could be as long as 100 questions.

[1] http://coffeelands.crs.org/wp-content/uploads/2012/04/Borderlands-Colombia-Baseline-Survey-FINAL.pdf
[2] http://www.fantaproject.org/.


Please let us know if you have any questions or need more information on our projects and we will be more than happy to tell you more about it
 

Topic locked