There are 7374 producer companies in India

There is increased interest among policymakers and practitioners on collectivization, as it can offer a platform for small and marginal farmers to negotiate with markets from a position of greater strength.  More and more producer companies (PCs) are being formed. Therefore, it is important to know their numbers, their distribution and health. Yet, as those working on producer companies are aware, disaggregated data on registered producer companies is not easily accessible.

Therefore, my colleague and I undertook an exercise to clean up and analyse data uploaded by the Registrar of Companies on all types of pubic and private companies registered until end of March 2019.

As of March 31, 2019, a total of 7,374 PCs have been registered in the country. Almost all states and union territories have registered some producer companies.  However, a large number of companies are concentrated in some states: for example, Maharashtra alone accounts for more than one quarter of all producer companies in India.

PCs by state

The above table shows state-wise distribution of producer companies registered until end of FY18-19.  92% of these companies are farmer producer companies (FPCs).

The 7374 registered producer companies have a combined Paid-Up Capital (PUC) of about Rs. 860 crore. As one would expect, there are a few companies with very large PUC and a large number of companies with very small PUC.  For example, the PUC of top 100 companies (Rs 587 crore) accounts for more than two thirds of the total PUC of all companies and at the other end, there are 189 companies with just Rs. 1000 or less PUC each. The median PUC is Rs. 1.06 lakh for all registered companies.

As part of this analysis, we also estimated the total number of farmers who are now members of FPCs and the number turned out to be quite large: 4.3 million.  There are roughly 90 million farming household in India, so 4.3 million represents roughly 5% of all farmers.  With continued formation of new PCs, this number is expected to increase.

We recently published an article on this in The Wire with more details: How Many Farmer Producer Companies Are There in India?

We are working on a comprehensive report, which will be released over the next few months.




How machines can learn gender bias

Several readers wrote to me saying that they found my previous post on algorithms propagating social inequality interesting. So here is a link on how machines (algorithms) can pick up gender bias in literature.

“This machine read 3.5 million books then told us what it thought about men and women” (from World Economic Forum):  VideoArticle


How everyday algorithms propagate social inequality

I used to have a colleague who was a Canadian engineer.  He wore an Iron Ring on his little finger as a reminder of the obligations and ethics associated with his profession. He explained to me that Iron Rings used to be made from the rubble of the Tacoma Narrows bridge which collapsed soon after construction due to a flaw in its design. Data scientists may soon need a similar ring – to remind them that their algorithms have a direct impact on people’s education, jobs, their lives.

Seeing the increasing use of various kinds of algorithms to make decisions about real people, I have been getting interested in the ethics (or their lack) embedded in such algorithms.  So I read with great interest the book ‘Weapons of Math Destruction’ by Cathy O’Neill.

Digital technology drives much of our modern life.  When shopping online, a recommendation generated by the site’s algorithms might lead us to an impulse purchase.  When browsing the internet, we see ads targeted to us based on our search history, nature of emails and online behaviours on various sites.  And we get charged different premium on car insurance based on our credit scores, and not just our driving history.

The author, Cathy O’Neill who’s a mathematician turned wall-street-quant turned data-scientist, points out that such algorithms embody many statistical issues:  One of the first statistical principles students are taught is to avoid conflating correlation with causation.  Just because two variables move together does not necessarily mean that one causes the other.  Correlations between ‘A’ and ‘B’, may be because A causes B, or B causes A, or C causes both A and B, etc. or the correlation may be entirely spurious.

Yet, this is exactly what Big Data algorithms do — by design.  They look for correlations but treat them as causation.  How strongly does living in a particular neighbourhood correlate with being credit worthy or being employment worthy?  What percent of people with certain types of names & addresses are likely to leave a new job within the first year?

Correlations between where a person lives and their credit worthiness are a reflection of their wealth and social factors:  In the US, they often relate to race, and in India, they often relate to caste and religion.  An algorithm deciding credit worthiness on the basis of one’s address will automatically incorporate caste and religion into its calculations.  Thus use of such algorithms for determining credit worthiness will inherently perpetuate social inequality.

Similarly, any algorithm calculating a new recruit’s chances of making it to senior management, will look for correlations among trajectories of past employees, and automatically incorporate gender and caste bias.

The book points out other statistical issues as well.  One school district in the US measured teacher performance based on average improvement in student scores during the school year.  One teacher discovered that his score varied dramatically year to year, based on different composition of students in class.  One year his class had a greater number of students at two extremes of academic performance – many students with learning disabilities and several with high academic performance.  In both cases, it was difficult to imagine significant improvement in student marks during the school year.  So he was rated poorly by the algorithm.  Next year, when he had a more ‘normal’ distribution of students, his score shot up dramatically.  One  reported year-to-year rating variation of 40 points, another reported 90 points (out of a 100)!

In another school district, a teacher who was considered an excellent teacher by students and the school alike, was fired due to a low relative score assigned by a computer algorithm. Her requests for clarification were dismissed. However, many years later, due to other similar cases, when the school district revealed parts of the algorithm, it became clear that her  low score was because other teachers were gaming the system by correcting students’ answer sheets.

Students in the school system needlessly lost excellent teachers due to a non-transparent algorithm which resembled phrenology more closely than a robust statistical analysis based on a good understanding of underlying dataset and its distribution patterns (e.g. knowing how composition of a class of students might affect learning ‘improvement’) and use of robust statistical methods.

Another issue:  Various companies target us by grouping us into clusters of ‘people like us’. As these customer segments become finer and finer, some category of products such as insurance lose their meaning altogether.  Insurance was originally designed such that a group of premium payers benefit collectively: Everyone pays a premium. Those who end up facing significant expenses or a catastrophic event gain compensation, while those who don’t still benefit from the peace of mind but not financially.  This basic premise of insurance products gets thrown out the window when the customer segment is so narrow as to be almost predictive. In such scenarios, insurance payers are simply betting against themselves and insurance as a product becomes almost meaningless.

Although such algorithms bring efficiency and lower costs, they lack many principles of good design. Most of these algorithms do not come with feedback processes which might lead to improvements over time. If a job candidate is denied a job, the algorithm doesn’t track that candidates meteoric rise to leadership positions in a different organisation.

There are also arguments for taming Big Data which are based on concerns about privacy and imposition of penalties for personal choices (e.g. some American companies coercing people to sign up for closely-monitored ‘wellness’ programs or pay extra, without any proof that either these programs work or that the ‘wellness’ measures they track are the relevant ones).

So here is a list of common issues with such algorithms:

  1. Unproven assumptions baked into the models
  2. Poor understanding of statistical methods (conflation of correlation with causality, cluster analysis, Simpson’s paradox, etc.)
  3. Lack of transparency which prevents us from knowing what criteria are used for decision-making and to challenge them
  4. Lack of feedback mechanism for improvement and catching errors
  5. No appeal process for those who are harmed (the algorithm’s judgement is final)
  6. No control of personal data (error correction) and privacy

Such warnings may seem a little extreme, especially in the Indian context, where we are only beginning to encounter such algorithms.  But as more and more aspects of our lives become directed by these algorithms, it is important for us to take notice now before the algorithms become too entrenched to change.

Cathy O’Neill offers a two-part solution:  She proposes regulatory oversight of algorithms to deter discrimination and exploitative practices of various kinds, just like many industries already are.  She also proposes that data scientists undertake a Hippocratic Oath.  Just like some Canada-trained engineers.



Farmer Producer Companies and Agrarian Distress

Over 7000 farmer producer companies (FPCs) have been promoted in the country since the enactment of Producer Companies Act.  A colleague and I are conducting a study of such companies.  We are finding that most such companies are under-capitalized.  Any many are simply defunct.

Instead of trying to address the issues, the policies continue to focus on launching more new companies.  Here’s my recent piece in TheWire on why FPCs are an important part of the solution to agrarian distress, but not the solution per se.

More to come on this topic… stay tuned

Of latitude and longitude

“We never used to go out or know about new things.  Now that we know about ‘line siri [SRI technique]’ we are practicing that.  The paddy production is much better now.”

“Now even if the men are away somewhere, we can go get fertilizer from the [Farmer Producer] company shop ourselves.”

These words spoken by women farmers in Lucknow district are emblemic of farmers I have met elsewhere in the country.  The women are thirsty for more information and access.  They toil away on family farms without acknowledgement of their labour and contribution.  Worse, they are not included (and in some cases actively excluded) from knowledge networks which might help them make better decisions about crop choice, cultivation practices, and markets.

The women talked candidly of their experiences and concerns.  The meeting was full of warmth, openness and enthusiasm, which I find to be invariably the case in meetings with women farmers across the country.

In this village, many farmers have come together to form a Farmer Producer Company (FPC).  The company has set up an inputs shop for its shareholders (~20% of whom are women) in a village about 1.5 km away.  Prior to the FPC shop, women had not been involved in the process of purchasing and picking up inputs from any shop; this was a role reserved for men (who have greater mobility).  They explained that now they could get information through awareness camps conducted by the NGO which was supporting the FPC.  Some women said that they find it very convenient to buy inputs from the shop and they were excited about learning new techniques like SRI (System of Rice Intensification).

The shareholders of this FPC are marginal farmers, with an average of 1.5 acres of landholding, growing paddy, wheat, mustard, potatoes and a few other crops.

Earlier in the day, my colleague and I had met the board of directors (all men; there was one female director, who was not available).  The discussion was animated and covered a wide range of topics, not dissimilar from other such discussions I’ve had in northern India. They commented on how the fertilizer in the market is highly adulterated and the distributors charge over and above the official MRP, often 50% more than the IFFCO MRP.  Not surprisingly, they explained how the creation of a Farmer Producer Company has been instrumental in getting access to good quality fertilizer at MRP when they need it.

The male farmers explained that when they had started the FPC, they had raised only a limited amount of equity capital.  They needed additional funds to buy good quality fertilizer in bulk. No bank or other formal sector entity would give them a loan.  After several attempts, the farmers took a loan from the Pradhan, who is also a member of the company.  They also organized loans from many other sources, which enabled them do proceed with their activities.

They feel frustrated by agriculture policies on several fronts.  For example, they say that the  government has not notified FPCs as being on par with farmer cooperatives when it comes to distribution of fertilizer. As a result, during peak season when fertilizer becomes available, it is first sent to cooperatives and the FPC is unable to get sufficient quantity.  They thought of buying during off-season but they don’t have enough working capital or storage capacity to store enough for all members.

The farmers grumbled about other interactions with bank officials:  The officials don’t know what a producer company is.  We had to tell them about FPCs.  Even then they didn’t want to support us because they did not have a circular.  The farmers claimed that one government circular stated that a particular scheme is available only to FPCs registered by SFAC.  “They don’t even know that SFAC does not register companies and only ROC (Registrar of Companies) does. We tried to convince them but they demanded a revised circular.

They gave more examples:  The FPC, under the guidance of the NGO-promoter, tried getting a license to procure at MSP (minimum support price), to become an “MSP centre” as they called it.

The FPC got the license and decided to open the centre for procurement.  After a few days of procurement they were informed that they must pay farmers for the procurement within 72 hours or face jail.  But they were expecting FCI (Food Corporation of India) to pay them about a month later. The head of the procurement centre was in a bind and felt distressed, saying ‘ab ya to jail ya fansi‘ (I am facing jail or suicide by hanging).  Somehow or the other, with the help of other producer companies and the promoters, well-wishers, etc, they managed to come up with the funds.  Not surprisingly, they decided to terminate all MSP procurement activities immediately.

The Chair of the FPC board lamented:  “How are marginal farmers supposed to come up with enough money to pay for all their entire produce within 72 hours?  We should be given advance like govt MSP centres are.”  

Perhaps a practical approach might be to allow FPCs to pay after receiving funds from FCI.  It is odd that these rules are being enforced without understanding the nature, capabilities and purpose of FPCs.

The farmers cited yet another example: They have recently received a circular requiring them to upload the geographical coordinates of their FPC office (geo-tagging of latitude and longitude) and a photo of their office on the website of the Registrar of Companies.  The farmers are relying on the NGO-promoter for compliance, despite the fact that the NGO’s project funding (and formal support activities) ended a few months ago.  They also pointed out that the forms and the website are in English for which too they need the NGO’s support.  The chair of the FPC board said “Unhi ka sahaara lenge (we will continue to rely on them)”; the NGO team member added quickly “jab tak yahaan hain” (until we are here).  I got the feeling that the farmers were referring to not only the NGO but the individual team members sitting among us, with whom they had built a trusting relationship.

In the course of my work, I have held similar discussions with farmers from different parts of the country.  Once we get over the initial hesitation, the discussions become open and candid.  This meeting was no different.  But it was unique in that most of the farmers’ laments were directed at policies which appear to be pro-farmers on the surface but are impractical, such as MSP procurement centre rules or expecting barely-literate marginal farmers to geo-tag their offices.

This particular group of farmers seemed to be well-informed and highly-engaged, and their FPC was quite active.  Yet they were unsure how they would manage after the exit of the NGO-promoter.

Update: The promoter of this FPC adds, “Today, ROC compliances for TATA or Reliance and a producer company situated at remote rural  location is same.  Recently ROC asked to update KYC of all BODs [Board of Directors] of companies, otherwise a heavy penalty was imposed.  Doing KYC of BODs means [that] each BOD had to upload his video and also whole process was linked with BOD mobile number. BOD received an OTP which he has to send to CA for uploading KYC documents.  In this process many BODs [who didn’t] have smart phone or not able to handle whole process got stuck and the result was ‘heavy penalty on FPC’. We are promoting these FPCs at remote locations to help small and marginal farmers, and we know about status of our country infrastructure and status of digitization.”

Explainer: Minimum Support Price (MSP)

There are many misconceptions about MSP, which I had written about in an earlier post. To help my students understand this better, I put together a short explainer on the topic, including some inputs from my colleague.  I’m sharing a link below, in case regular readers of this blog find it useful too.  Please do share your feedback in the comments section below.

MSP Explainer

The deep silence of integrity

There was pin-drop silence in the room.  Occasionally, I could hear someone clearing his throat hesitantly or the scratching of pens and pencils on paper.  We were all afraid to lift our eyes to look around the room or even breathe loudly.

It was a mid-term test and the professor had left us students alone in the room. He had distributed the question papers and said that he had to do something urgent and would be back in twenty minutes.  He said that we should work on our test; he is trusting us not to talk or cheat during the exam.  And then, surprisingly, he simply left.

Maybe he was standing and listening to us right outside the door the whole time.  Maybe he actually had something urgent which needed to be done immediately.  Maybe he was conducting an experiment on whether students would cheat on a test if there was no invigilator in the room.  Or maybe he had simply made a bet with a colleague.

Whatever the case, he left his class of about 40 students alone in the examination hall, for a test which was worth probably 30% of the course grade.  This was a novel experience for me, as a young undergraduate at the University of California, at Berkeley.  Never had any teacher left the class during an exam.

He was gone for about 20 minutes.  To me, it felt like an eternity.  I had never seen a bunch of undergrads observe such pin-drop silence even during the most fascinating lectures.  For the next 20 minutes, I did not hear a single voluntary human sound.  Even the clearing of someone’s throat sounded apologetic and hesitant.  Such was the burden of integrity placed on us, that we complied half out of fear of being accused of cheating, and half out of pride of displaying impeccable integrity.

I contrast this experience of two decades ago with a recent experience.  A group of about 40 newly admitted students sitting for a test worth about 20% of their course grade.  Despite the invigilator repeatedly reminding students to not talk during the test, there is recurrent murmuring.  The invigilator has to leave the room for one minute to facilitate a swap with a professor in another room.  When the professor enters, he finds several students talking. He admonishes them and is forced to change the seat of a student and still the murmuring continues.

When I discussed with students why they cheated, I received startling responses.  One student claimed that he didn’t cheat.  Surprised, I asked him, “You didn’t talk at all in the class?”

He replied, “Yes, I talked but I didn’t cheat.”

“What did you talk about?”

“I asked her about question 6.”

Incredulous, I asked, “Isn’t that cheating?”

“No. I only asked one question.  Cheating is when you copy the whole test. And, she said that she didn’t know the answer.  So I didn’t cheat.”

I almost laughed.  I had to explain that whether it was one question or all, whether he got a useful response or not, his actions constituted cheating.

His response?  “I am from UP.  This is how things are done there.”

Another student offered a variant of this excuse:  “I didn’t know that this is not allowed at Azim Premji University. This is my first test here.”  One student who hadn’t cheated defended others by claiming that it was the invigilator’s responsibility to “snatch away their papers” instead of only telling them to not talk and cheat, thus blaming the invigilator for lack of vigilance.

Of course, the majority of incoming students hadn’t cheat and only a few had. But the fact that many who hadn’t cheated saw cheating as a matter of process compliance and not integrity is both surprising and disheartening.  Today’s students are going to be the professionals of tomorrow. They are going to be the professionals who power the future economy, who go on to become leaders of small and large companies, and in the my university’s case, become development practitioners working to tackle issues of climate change, farmers’ livelihoods, public health and social justice.

In setting up Indian students on an almost maniacal pursuit of high marks, we have taught them that not only must they do ‘whatever it takes’ to get those high marks, but that it is perfectly acceptable to do so.

In our masters program, we have just two years to bring about a dramatic change in the mindset of students who come with such beliefs, to inculcate integrity and commitment, not only through fear of being caught, but by understanding the link between personal integrity and societal outcomes.  We hope to help these students see that moral corruption is not just about the large scams in the newspapers. It is also the traffic cops taking a bribe from motorcyclists without documents; the doctors prescribing procedures we don’t need; and some students cheating in exams in the hope of better jobs.

When comparing this recent experience with that from an un-invigilated test of two decades ago, I can’t help but hope that in the two years we have them, we are able to help students realize that they don’t need an invigilator to stay honest.  And that leading social change is not always about loud raucous speeches, long dharnas and high-visibility innovation; often leading social change starts with personally demonstrating the deep silence of integrity arising from within.