I like writing poetry and I am not good at it at all

Posted on by Anand Philip

So I threw some of these lines at Bard and true to its name, it made stuff that is better than my lines. Still not sure if any of it counts as poetry but there are definitely a line or two or a metaphor or two here that are interesting.

What I wrote:

Read Nietzsche the first time to appear well read and find confusion
The second time to appear serious and find epiphanies
The third time for yourself and reject the overman
 

What the LLMs rewrote:

Nietzsche’s pages turn,
Confusion sparks enlightenment,
Overman dismissed.1

First, a borrowed cloak,
Nietzsche struts, impressing crowds,
Empty, echoing.

Then, a seeker’s beard,
Epiphanies sought in smoke,
Ashes in his hand.

Finally, unmasked,
He laughs, tears off the false skin,
Free, and only man.2

Tried Nietzsche to swagger, impress the crew,
Brain scrambled, left me scratching my shoe.
Second go, all earnest, brow furrowed deep,
“Aha!” moments came, then went to sleep.
Third and final, just me and the page,
“Übermensch” baloney, turning the cage.
My own way to go, the hype I dismiss,
Nietzsche’s a puzzle, not some hip abyss.3
 

First dive into Nietzsche for some street cred,
Got lost in the jumble, scratching my head.

Round two, looking serious, seeking insight,
Epiphanies popping up, left and right.

Third time’s a charm, just for myself,
Overman? Nah, put that book on the shelf.
Finding my path, rejecting the hype,
Nietzsche, my guide, in this philosophical tripe.4

Thought Nietzsche cool, like shades and a scowl,
Read, got lost, felt like a clueless owl.
Tried again, serious, highlighter in hand,
“Deep thots,” I’d shout, then couldn’t stand.

Third time, alone, book open and worn,
Overman stuff? Nah, more like a yawn.
My own truth’s out there, beyond borrowed gleam,
Nietzsche’s a teacher, not some grand scheme.5

(Verse 1)6
Caught a freight train down to wisdom’s depot, Nietzsche in my pack
Heard the hipsters talkin’, thought I’d join the existential act
Cracked the spine, let words like shadows crawl across my mind
But meaning danced like dust devils, never could I pin it down

(Chorus)
First time through, a borrowed mask, words I didn’t own
Spouted wisdom from the rooftops, just another crow cawin’ alone
Second time, all furrowed brow, highlighter in my hand
Epiphanies like lightning bugs, sparklin’ on the promised land

(Verse 2)
But lightning fades, and truth ain’t found in borrowed sparks
So I stripped it back, raw wood and nails, no fancy, gilded arks
Read for me, not crowds, not critics with their sharpened quills
Just a man unmasking shadows, climbin’ his own personal hills

(Chorus)
Third time round, the book worn thin, spine crackin’ with each turn
No more Übermensch charades, let the embers of my spirit burn
I ain’t no superman, don’t crave Olympus’ golden throne
Just lookin’ for a path of my own, where the wind whispers in my bones

(Bridge)
So thanks, Nietzsche, for the storm, the tangled mountain trail
You showed me wisdom ain’t a summit, but a fire that never fails
I may have borrowed your compass, but the stars I follow now are mine
Each scar a map, each breath a song, this journey’s truly divine

(Chorus)
First time blind, second time blurred, but now the focus clear
No borrowed wings to lift me up, my feet planted right here
So long, Zarathustra, your shadows fade in the mornin’ light
This path’s mine, and I’m walkin’ on, the stars as my sight


  1. Brief and bloody well summarized ↩︎
  2. My favorite: borrowed cloak, seekers beard, ashes in hand. ↩︎
  3. I do like a hip abyss ↩︎
  4. This was weak, i know. but of the 15 odd poems i generated this was in the top five ↩︎
  5. A teacher, not a scheme. damn good ↩︎
  6. and finally as a bob dylan song, can’t be sung to Mr. Tamborine man’s tune tho ↩︎

A conversation about odorless blue thoughts

Posted on by Anand Philip

The following is a conversation with a LLM chatbot Llama, which is just funny. using the model llava-v1.5-7b-q4

User: what is a category error in logic and give some examples as well as examples of what is not a category error although it looks like it

Llama: In logic, a "category error" refers to an argument that appears valid at first glance but actually violates the rules or structure of a particular logical category. It's essentially when someone tries to apply one type of reasoning to another situation where it doesn't belong, resulting in a fallacy.

An example of a category error is trying to use deductive reasoning (which relies on general principles and specific premises) to prove an inductive conclusion (where the truth of the conclusion follows from the truth of its individual instances). This would be considered a category error because deduction and induction are different logical methods, each with their own rules for drawing conclusions.

Another example is using the fallacy of false analogy or comparing apples to oranges (using an inappropriate comparison) instead of comparing them within the same category. For instance, you might say that a politician's actions are like those of a thief because they both break laws; however, this would be a category error since politicians and thieves belong to different categories with distinct characteristics and rules for their respective behaviors.

On the other hand, although it may seem similar at first glance, an example that is not a category error is comparing apples (fruit) to oranges (fruit). Although both are fruits, they have different properties like taste, texture, and nutritional content. It's important to recognize when we're using the correct logical method for drawing conclusions based on available evidence or reasoning about a particular situation.

User: is the following sentence semantically accurate? "Odorless blue thoughts run lopingly"

Llama: The sentence "Odorless blue thoughts run lopingly" is not semantically accurate, as it contains several grammatical and semantic errors. Firstly, there's a missing article ("the") before the word "blue," which should be "The." Secondly, the verb "run" doesn't agree with its subject "thoughts" in number; it should either be singular or plural to maintain consistency (either "Run lopingly" or "Runn lopingly"). Lastly, there is no clear connection between odorlessness and running.

To make the sentence semantically accurate, you could rephrase it as: "The absence of scent in blue thoughts allows them to run freely." This version maintains grammatical correctness while also conveying a meaningful message about how thoughts can move or flow without any obstruction.

User: what could the phrase "Odorless blue thoughts run lopingly" mean?

Llama: The phrase "Odorless blue thoughts run lopingly" appears to be an intentionally nonsensical and humorous construct, likely meant for entertainment purposes rather than conveying a meaningful message. It combines elements from different domains (odorlessness of the color blue, running as an action) in a way that doesn't make sense semantically or logically. The intended effect is to evoke amusement and curiosity by presenting an absurd phrase rather than adhering to any strict rules of language or logic.

User: What does the phrase "The absence of scent in blue thoughts allows them to run freely." mean

Llama: The phrase "The absence of scent in blue thoughts allows them to run freely" is a revised and semantically accurate version of the original nonsensical sentence you provided. It suggests that without any distractions or obstructions (represented by the lack of odor), one's thoughts can move freely, unencumbered by external factors like scent or other sensory stimuli. This metaphorically connects the idea of running as a physical action with the freedom and clarity that comes from having undistracted thought processes.

Featured Image is by Meenakshi Payal from Flickr. It shows a Bharatnatyam Performance by Sneha Chakradhar at India International Center

Stochastic parrots no more

Posted on by Anand Philip

Tech is heavily invested in giant language models.

This is because, with transfer learning, language models suddenly started being able to do a LOT more. And a lot better.

A lot of high citation publications have also been an outcome of this exploration of language models

For most parts in NLP engineering, LMs went from being cute to being the standard model from which you build solutions in the last three years. And it’s not just in NLP, the language model approach (use large amounts of data to train a network to do specific tasks, and some approximation of the underlying first principles will be approximately encoded into the model) seems to work well for images as well (stable diffusion).

And given that all of deep learning and the engineering approach to AI seem to be approximate/mimic till you make it, this is a powerful learning.

For academics, in a sense, iterative improvement, criticism, existential threats etc. are part of the publish-perish cycle. For the engineer, iterative improvement is, but it is driven by entirely different incentives.

When the ethics people started talking about bias, the engineering world was not very happy about it, well no one likes the police. More so because bias is not easy to tackle, its not easily quantified, its not something you can fix with a software update. And because there doesn’t seem to be any consensus yet about how to deal with it.

I think there are distinct parallels here from the early days of medical ethics and evidence based medicine. No one likes to hear that their life’s work is actually harming people despite having honorable intentions. We’d all like to be told that well, this is OK, you were doing the right thing, and we can fix it, easy.

But as the decades of literature in medical ethics and the history of clinical medical practice shows, ethics isn’t easy but is also not optional. The medical world was brought kicking and screaming into getting its shit together and behaving.

 

What is semantic search and why do we want it

Posted on by Anand Philip

You’re searching for poems that talk about cutlery. Because you can. Also because a lot of times you don’t remember the precise detail or term used in a document that you want to find. And also because compiling a list of poems about our relationship with food-ingestion-tools would be fun.

Maybe it’s not poetry you care about but the name of the company your boyfriend works at which he told you about three months ago but cant find on your Whatsapp. And obviously, it would be super awkward to ask him now.

We have disparate search needs is what I am trying to say.

Lexical search: or how most search works

Definitions:
Query: The text we enter into search fields when we want stuff found

Document : The file, website, chat, image, meme stash or entity that has the terms or could have the the right answers to your search

Corpus: The collection of all the documents that need to be searched across. (your whole website for example)
On with the show!

The standard way in which we do search is by matching the words or terms in a query with the terms in the corpus. So if you search for poems about cutlery  the search will look for documents that have the words poems and/or cutlery and try to find ones that contain both the words.

Imagine, though, that the program has to scan through every document every time someone searches for something, hunting for instances of the word from scratch. This would be pretty time-consuming and also slightly stupid. So we need a way to store what words are in what documents.

Besides just matching or finding the word, we also need some way of knowing which document matches it better. This is called ranking. To solve both these problems  people came up with the idea of indexes (or indices, if you’re a Latin snob).

What an index does is, it look over all the documents in a corpus, and creates a brief representation of the content of that document. So instead of searching over the whole document, now you just search over the representations. These representations also have some way of ranking the quality of the fit between the query and the document.

An example of such a representation1 is TF-IDF (Term Frequency–Inverse Document Frequency). What this does is,

  1. Lists all the words in each of the documents in a corpus and how frequently the occur in a document (term frequency).
  2. Calculates how common or how rare of each of these words are across the whole corpus. The common words get a score of 0, or close to zero and the rare words get high scores. 2

Why do we do this? In most documents, the words that appear most frequently would be things like the, and, is and stuff that doesn’t add any discriminatory power to the retrieval.

When I search for poems with cutlery  there are going to be a whole lotta  poems with the word with in them, so I can’t use with to rank the documents in any way. But the word cutlery probably appears in a small number of poems, so it is a good word for ranking the results.

So now your index of poems doesn’t just have the whole text of the poem but a list of the non-zero scoring words along with the scores, which allows us to get the top matches.

The most used and probably the best of these TF-IDF  algorithms is the  Okapi BM25  family of algorithms. They are absolutely fantastic in finding you the documents that are best matching the words you search for.

But what if you don’t know the precise thing you are searching for? This is also a problem as old as misplaced keys. One way to solve this would be to have some kind of a thesaurus. You search for cutlery  and we expand the terms to include all the synonyms of the word. This is called query expansion. So instead of sending the term cutlery to the index, we can send cutlery, knife, fork, teaspoon and see what the index gives us.

But that means now you need to figure out which words you want to find synonyms for, how to store them, how to update them periodically etc. And what if you need to find synonyms of phrases, not words, eg. our relationship with modern day digital technology. That would make very large search term if we expanded each word. Still, this is a pretty damn good method and is used widely and is what Google uses to return close matches.

Another problem could be that you might misspell the word, or that your way of spelling antidisestablishmentarianism3 is different from the way it appears in the text, say “anti-dis-establishmentarianizm” or whatever.  The usual solution to this is to add some “fuzzy” logic or approximate matching. This works by searching for words that are off or different by a few letters. Unfortunately this means that if you search for costly coat cuts,  you’ll also get results for costly cat guts 4.  Still, this too is pretty damn good and is usually  a part of most search systems.

You could also offer spelling correction, as google often does. But problems with autocorrect are well known

I am compelled to point to yet another problem: Besides synonyms, there are words that are structurally similar, or stem from the same word. These too might make a good fit or a bad fit for a particular search query based on the context, for eg. if you’re searching for load bearing walls, it might be good to get results for load bearing tiles, but not good to get bear loading 5

Anyway, so what we need is a way to represent text that can account for

  1. Variations in the way a word is written
  2. Contextual meaning of words (A coat of paint vs a coat of mink)
  3. Spelling mistakes
  4. Synonyms

All these things are present in the way we write and store information. So if you were to take a sufficiently large sample of things people have written, you would find all of these things in this sample. This idea is what gives us semantic search.

Enter AI!

Created with Dall-E

Kidding, AI is a marketing myth, who wants to buy “statistical-learning models”6?

But that is what we do, we take a large corpus of text, and then derive a statistical model of the relationships between words in a multi dimensional space. And we do this by masking text and getting a model to fill in the blanks. No, seriously, that’s it. This is the basis of all LLMs.

What this does is, make a model memorize and therefore extract all the million ways in which any given word can be used in a language and also how each word in the corpus is related to every other word. This is not something we can manually compute and that is why we throw artificial neural networks at it.

Now, most LLMs and other things that are called AI these days don’t stop here, and have to predict specific things. But if you take off those prediction bits from the model what you have left is called an embedding. Which is a multi-dimensional representation of whatever it is that you trained the thing to do. Engineers, like many of us, suffering from grievous mathematics envy, call these representations vectors.

The way these work is that words and documents that are similar in meaning are represented closer to each other, or clustered. But this happens over many dimensions, so it’s not a simple 1 or 2 D relationship it captures.

See how the word noticed is close to, and connected to all the words that are similar in meaning. The above is a screenshot of one such embedding. I strongly urge you to explore this. Link: Word2Vec Tensorboard

Another tool to understand how words can be represented in multiple dimensions is this embedding explorer from Edwin Chen.

What is most interesting to me is that this rich and pretty accurate representation is produced using something as simple as fill-in-the-blanks.

Searching for things using the information implicitly encoded in these vector spaces is called semantic search by software and ML people.7

So if you don’t want to manually manage everything from synonyms to contextual meaning, you could take a vector space trained on a very large corpus, convert your corpus into those kind of vectors, and then search over those vectors. This conversion just borrows the relationships that are already discovered by the model, which means its representations will be much richer than the ones in your corpus.8

These vector spaces are not like the indexes we spoke of earlier, they don’t have any explicit way of saying word x is present in document z and the score is high.

The search works by identifying documents that are “close” to the query in the vector space. The most commonly used metric to determine this distance is cosine similarity . But even if you can calculate the cosine distance between two words, doing this calculation for every word in your corpus against every word in the query would be mind-numbingly boring and also time and resource consuming. So we need some way to find the location in the vector space that is most likely to have good results, and get there quickly.

This is called approximate nearest neighbor search, and there are a bunch of great algorithms that have come up in the last few years that let you search over humongous datasets really fast.

The ones I like the most are Neighborhood Graph Trees and Hierarchical Navigable Small Worlds (HNSW)

Both of them work by dividing up your data into small clusters, or trees and then searching in those trees or clusters. But really, it’s a lot more complicated than that and to be honest I don’t really understand graph theory enough to know what they do. But what they do they do do well.

This might remind you of indexes, and that is what these create, just, vector and tree representations, not words.

image from Pinecone’s guide to semantic search, linked below

Summary

  • You embed your corpus onto an existing language model’s vector space
  • You then index your embedding using a ANN algorithm like NGT or HNSW
  • And then you can search your corpus using semantic querying.

A demo

If you’ve managed to stay here this long, I urge you to explore the subject using a live demo. MoodMuse is a small app I made to learn and demonstrate the topic. It finds you poetry, based on the meaning of your words, not the terms alone.

Thanks for reading.

This cat does not exist. Created with https://deepai.org

Further reading and resources

Footnotes

  1. This is not the most commonly used representation, sometimes you just store the title of the document, somethings you store a description, it depends on the need of the system and users. ↩︎
  2. Engineers really love logarithms. We should too ↩︎
  3. I am, of course a disestablishmentarianist and several centuries behind on news ↩︎
  4. If you’re into coat cuts. Whatever they are, no judgement here. ↩︎
  5. I know, my metaphors need work. They are unemployed ↩︎
  6. or “Function Approximators in high dimensional manifolds with surprising combinatorial capabilities” or “applied approximation theory” ↩︎
  7. Complicating the matters is that before there were large neural network based embeddings, there were the semantic web people and the ontology people and other people who wanted to solve this using structured representation of text. So software people are kinda stealing the term a little bit. More info: A survey in semantic search technologies ↩︎
  8. Thankfully there is a huge community of NLP nerds who like making these kinds of models easy to use (if you can Python, that is). ↩︎

Sultana’s Dream (1905) by Rokeya Sakhawat Hossain

Posted on by Anand Philip

This story was originally published in The Indian Ladies’ Magazine, Madras, 1905. Taken from WikiSource.

One evening I was lounging in an easy chair in my bedroom and thinking lazily of the condition of Indian womanhood. I am not sure whether I dozed off or not. But, as far as I remember, I was wide awake. I saw the moonlit sky sparkling with thousands of diamond-like stars, very distinctly.

All on a sudden a lady stood before me; how she came in, I do not know. I took her for my friend, Sister Sara.

‘Good morning,’ said Sister Sara. I smiled inwardly as I knew it was not morning, but starry night. However, I replied to her, saying, ‘How do you do?’

‘I am all right, thank you. Will you please come out and have a look at our garden?’

I looked again at the moon through the open window, and thought there was no harm in going out at that time. The men-servants outside were fast asleep just then, and I could have a pleasant walk with Sister Sara.

I used to have my walks with Sister Sara, when we were at Darjeeling. Many a time did we walk hand in hand and talk light-heartedly in the botanical gardens there. I fancied, Sister Sara had probably come to take me to some such garden and I readily accepted her offer and went out with her.

When walking I found to my surprise that it was a fine morning. The town was fully awake and the streets alive with bustling crowds. I was feeling very shy, thinking I was walking in the street in broad daylight, but there was not a single man visible.

Some of the passers-by made jokes at me. Though I could not understand their language, yet I felt sure they were joking. I asked my friend, ‘What do they say?’

‘The women say that you look very mannish.’

‘Mannish?’ said I, ‘What do they mean by that?’

‘They mean that you are shy and timid like men.’

‘Shy and timid like men?’ It was really a joke. I became very nervous, when I found that my companion was not Sister Sara, but a stranger. Oh, what a fool had I been to mistake this lady for my dear old friend, Sister Sara.

She felt my fingers tremble in her hand, as we were walking hand in hand.

‘What is the matter, dear?’ she said affectionately. ‘I feel somewhat awkward,’ I said in a rather apologizing tone, ‘as being a purdahnishin woman I am not accustomed to walking about unveiled.’

‘You need not be afraid of coming across a man here. This is Ladyland, free from sin and harm. Virtue herself reigns here.’

By and by I was enjoying the scenery. Really it was very grand. I mistook a patch of green grass for a velvet cushion. Feeling as if I were walking on a soft carpet, I looked down and found the path covered with moss and flowers.

‘How nice it is,’ said I.

‘Do you like it?’ asked Sister Sara. (I continued calling her ‘Sister Sara,’ and she kept calling me by my name).

‘Yes, very much; but I do not like to tread on the tender and sweet flowers.’

‘Never mind, dear Sultana; your treading will not harm them; they are street flowers.’

‘The whole place looks like a garden,’ said I admiringly. ‘You have arranged every plant so skillfully.’

‘Your Calcutta could become a nicer garden than this if only your countrymen wanted to make it so.’

‘They would think it useless to give so much attention to horticulture, while they have so many other things to do.’

‘They could not find a better excuse,’ said she with smile.

I became very curious to know where the men were. I met more than a hundred women while walking there, but not a single man.

‘Where are the men?’ I asked her.

‘In their proper places, where they ought to be.’

‘Pray let me know what you mean by “their proper places”.’

‘O, I see my mistake, you cannot know our customs, as you were never here before. We shut our men indoors.’

‘Just as we are kept in the zenana?’

‘Exactly so.’

‘How funny,’ I burst into a laugh. Sister Sara laughed too.

‘But dear Sultana, how unfair it is to shut in the harmless women and let loose the men.’

‘Why? It is not safe for us to come out of the zenana, as we are naturally weak.’

‘Yes, it is not safe so long as there are men about the streets, nor is it so when a wild animal enters a marketplace.’

‘Of course not.’

‘Suppose, some lunatics escape from the asylum and begin to do all sorts of mischief to men, horses and other creatures; in that case what will your countrymen do?’

‘They will try to capture them and put them back into their asylum.’

‘Thank you! And you do not think it wise to keep sane people inside an asylum and let loose the insane?’

‘Of course not!’ said I laughing lightly.

‘As a matter of fact, in your country this very thing is done! Men, who do or at least are capable of doing no end of mischief, are let loose and the innocent women, shut up in the zenana! How can you trust those untrained men out of doors?’

‘We have no hand or voice in the management of our social affairs. In India man is lord and master, he has taken to himself all powers and privileges and shut up the women in the zenana.’

‘Why do you allow yourselves to be shut up?’

‘Because it cannot be helped as they are stronger than women.’

‘A lion is stronger than a man, but it does not enable him to dominate the human race. You have neglected the duty you owe to yourselves and you have lost your natural rights by shutting your eyes to your own interests.’

‘But my dear Sister Sara, if we do everything by ourselves, what will the men do then?’

‘They should not do anything, excuse me; they are fit for nothing. Only catch them and put them into the zenana.’

‘But would it be very easy to catch and put them inside the four walls?’ said I. ‘And even if this were done, would all their business – political and commercial – also go with them into the zenana?’

Sister Sara made no reply. She only smiled sweetly. Perhaps she thought it useless to argue with one who was no better than a frog in a well.

By this time we reached Sister Sara’s house. It was situated in a beautiful heart-shaped garden. It was a bungalow with a corrugated iron roof. It was cooler and nicer than any of our rich buildings. I cannot describe how neat and how nicely furnished and how tastefully decorated it was.

We sat side by side. She brought out of the parlour a piece of embroidery work and began putting on a fresh design.

‘Do you know knitting and needle work?’

‘Yes; we have nothing else to do in our zenana.’

‘But we do not trust our zenana members with embroidery!’ she said laughing, ‘as a man has not patience enough to pass thread through a needlehole even!’

‘Have you done all this work yourself?’ I asked her pointing to the various pieces of embroidered teapoy cloths.

‘Yes.’

‘How can you find time to do all these? You have to do the office work as well? Have you not?’

‘Yes. I do not stick to the laboratory all day long. I finish my work in two hours.’

‘In two hours! How do you manage? In our land the officers, – magistrates, for instance – work seven hours daily.’

‘I have seen some of them doing their work. Do you think they work all the seven hours?’

‘Certainly they do!’

‘ No, dear Sultana, they do not. They dawdle away their time in smoking. Some smoke two or three choroots during the office time. They talk much about their work, but do little. Suppose one choroot takes half an hour to burn off, and a man smokes twelve choroots daily; then you see, he wastes six hours every day in sheer smoking.’

We talked on various subjects, and I learned that they were not subject to any kind of epidemic disease, nor did they suffer from mosquito bites as we do. I was very much astonished to hear that in Ladyland no one died in youth except by rare accident.

‘Will you care to see our kitchen?’ she asked me.

‘With pleasure,’ said I, and we went to see it. Of course the men had been asked to clear off when I was going there. The kitchen was situated in a beautiful vegetable garden. Every creeper, every tomato plant was itself an ornament. I found no smoke, nor any chimney either in the kitchen — it was clean and bright; the windows were decorated with flower gardens. There was no sign of coal or fire.

‘How do you cook?’ I asked.

‘With solar heat,’ she said, at the same time showing me the pipe, through which passed the concentrated sunlight and heat. And she cooked something then and there to show me the process.

‘How did you manage to gather and store up the sun-heat?’ I asked her in amazement.

‘Let me tell you a little of our past history then. Thirty years ago, when our present Queen was thirteen years old, she inherited the throne. She was Queen in name only, the Prime Minister really ruling the country.

‘Our good Queen liked science very much. She circulated an order that all the women in her country should be educated. Accordingly a number of girls’ schools were founded and supported by the government. Education was spread far and wide among women. And early marriage also was stopped. No woman was to be allowed to marry before she was twenty-one. I must tell you that, before this change we had been kept in strict purdah.’

‘How the tables are turned,’ I interposed with a laugh.

‘But the seclusion is the same,’ she said. ‘In a few years we had separate universities, where no men were admitted.’

‘In the capital, where our Queen lives, there are two universities. One of these invented a wonderful balloon, to which they attached a number of pipes. By means of this captive balloon which they managed to keep afloat above the cloud-land, they could draw as much water from the atmosphere as they pleased. As the water was incessantly being drawn by the university people no cloud gathered and the ingenious Lady Principal stopped rain and storms thereby.’

‘Really! Now I understand why there is no mud here!’ said I. But I could not understand how it was possible to accumulate water in the pipes. She explained to me how it was done, but I was unable to understand her, as my scientific knowledge was very limited. However, she went on, ‘When the other university came to know of this, they became exceedingly jealous and tried to do something more extraordinary still. They invented an instrument by which they could collect as much sun-heat as they wanted. And they kept the heat stored up to be distributed among others as required.

‘While the women were engaged in scientific research, the men of this country were busy increasing their military power. When they came to know that the female universities were able to draw water from the atmosphere and collect heat from the sun, they only laughed at the members of the universities and called the whole thing “a sentimental nightmare”!’

‘Your achievements are very wonderful indeed! But tell me, how you managed to put the men of your country into the zenana. Did you entrap them first?’

‘No.’

‘It is not likely that they would surrender their free and open air life of their own accord and confine themselves within the four walls of the zenana! They must have been overpowered.’

‘Yes, they have been!’

‘By whom? By some lady-warriors, I suppose?’

‘No, not by arms.’

‘Yes, it cannot be so. Men’s arms are stronger than women’s. Then?’

‘By brain.’

‘Even their brains are bigger and heavier than women’s. Are they not?’

‘Yes, but what of that? An elephant also has got a bigger and heavier brain than a man has. Yet man can enchain elephants and employ them, according to their own wishes.’

‘Well said, but tell me please, how it all actually happened. I am dying to know it!’

‘Women’s brains are somewhat quicker than men’s. Ten years ago, when the military officers called our scientific discoveries “a sentimental nightmare,” some of the young ladies wanted to say something in reply to those remarks. But both the Lady Principals restrained them and said, they should reply not by word, but by deed, if ever they got the opportunity. And they had not long to wait for that opportunity.’

‘How marvelous!’ I heartily clapped my hands. ‘And now the proud gentlemen are dreaming sentimental dreams themselves.’

‘Soon afterwards certain persons came from a neighbouring country and took shelter in ours. They were in trouble having committed some political offense. The king who cared more for power than for good government asked our kind-hearted Queen to hand them over to his officers. She refused, as it was against her principle to turn out refugees. For this refusal the king declared war against our country.

‘Our military officers sprang to their feet at once and marched out to meet the enemy. The enemy however, was too strong for them. Our soldiers fought bravely, no doubt. But in spite of all their bravery the foreign army advanced step by step to invade our country.

‘Nearly all the men had gone out to fight; even a boy of sixteen was not left home. Most of our warriors were killed, the rest driven back and the enemy came within twenty-five miles of the capital.

‘A meeting of a number of wise ladies was held at the Queen’s palace to advise as to what should be done to save the land. Some proposed to fight like soldiers; others objected and said that women were not trained to fight with swords and guns, nor were they accustomed to fighting with any weapons. A third party regretfully remarked that they were hopelessly weak of body.

‘”If you cannot save your country for lack of physical strength,” said the Queen, “try to do so by brain power.”

‘There was a dead silence for a few minutes. Her Royal Highness said again, “I must commit suicide if the land and my honour are lost.”

‘Then the Lady Principal of the second university (who had collected sun-heat), who had been silently thinking during the consultation, remarked that they were all but lost, and there was little hope left for them. There was, however, one plan which she would like to try, and this would be her first and last efforts; if she failed in this, there would be nothing left but to commit suicide. All present solemnly vowed that they would never allow themselves to be enslaved, no matter what happened.

‘The Queen thanked them heartily, and asked the Lady Principal to try her plan. The Lady Principal rose again and said, “before we go out the men must enter the zenanas. I make this prayer for the sake of purdah.” “Yes, of course,” replied Her Royal Highness.

‘On the following day the Queen called upon all men to retire into zenanas for the sake of honour and liberty. Wounded and tired as they were, they took that order rather for a boon! They bowed low and entered the zenanas without uttering a single word of protest. They were sure that there was no hope for this country at all.

‘Then the Lady Principal with her two thousand students marched to the battle field, and arriving there directed all the rays of the concentrated sunlight and heat towards the enemy.

‘The heat and light were too much for them to bear. They all ran away panic-stricken, not knowing in their bewilderment how to counteract that scorching heat. When they fled away leaving their guns and other ammunitions of war, they were burnt down by means of the same sun-heat. Since then no one has tried to invade our country any more.’

‘And since then your countrymen never tried to come out of the zenana?’

‘Yes, they wanted to be free. Some of the police commissioners and district magistrates sent word to the Queen to the effect that the military officers certainly deserved to be imprisoned for their failure; but they never neglected their duty and therefore they should not be punished and they prayed to be restored to their respective offices.

‘Her Royal Highness sent them a circular letter intimating to them that if their services should ever be needed they would be sent for, and that in the meanwhile they should remain where they were. Now that they are accustomed to the purdah system and have ceased to grumble at their seclusion, we call the system “Mardana” instead of “zenana”.’

‘But how do you manage,’ I asked Sister Sara, ‘to do without the police or magistrates in case of theft or murder?’

‘Since the “Mardana” system has been established, there has been no more crime or sin; therefore we do not require a policeman to find out a culprit, nor do we want a magistrate to try a criminal case.’

‘That is very good, indeed. I suppose if there was any dishonest person, you could very easily chastise her. As you gained a decisive victory without shedding a single drop of blood, you could drive off crime and criminals too without much difficulty!’

‘Now, dear Sultana, will you sit here or come to my parlour?’ she asked me.

‘Your kitchen is not inferior to a queen’s boudoir!’ I replied with a pleasant smile, ‘but we must leave it now; for the gentlemen may be cursing me for keeping them away from their duties in the kitchen so long.’ We both laughed heartily.

‘How my friends at home will be amused and amazed, when I go back and tell them that in the far-off Ladyland, ladies rule over the country and control all social matters, while gentlemen are kept in the Mardanas to mind babies, to cook and to do all sorts of domestic work; and that cooking is so easy a thing that it is simply a pleasure to cook!’

‘Yes, tell them about all that you see here.’

‘Please let me know, how you carry on land cultivation and how you plough the land and do other hard manual work.’

‘Our fields are tilled by means of electricity, which supplies motive power for other hard work as well, and we employ it for our aerial conveyances too. We have no rail road nor any paved streets here.’

‘Therefore neither street nor railway accidents occur here,’ said I. ‘Do not you ever suffer from want of rainwater?’ I asked.

‘Never since the “water balloon” has been set up. You see the big balloon and pipes attached thereto. By their aid we can draw as much rainwater as we require. Nor do we ever suffer from flood or thunderstorms. We are all very busy making nature yield as much as she can. We do not find time to quarrel with one another as we never sit idle. Our noble Queen is exceedingly fond of botany; it is her ambition to convert the whole country into one grand garden.’

‘The idea is excellent. What is your chief food?’

‘Fruits.’

‘How do you keep your country cool in hot weather? We regard the rainfall in summer as a blessing from heaven.’

‘When the heat becomes unbearable, we sprinkle the ground with plentiful showers drawn from the artificial fountains. And in cold weather we keep our room warm with sun-heat.’

She showed me her bathroom, the roof of which was removable. She could enjoy a shower bath whenever she liked, by simply removing the roof (which was like the lid of a box) and turning on the tap of the shower pipe.

‘You are a lucky people!’ ejaculated I. ‘You know no want. What is your religion, may I ask?’

‘Our religion is based on Love and Truth. It is our religious duty to love one another and to be absolutely truthful. If any person lies, she or he is….’

‘Punished with death?’

‘No, not with death. We do not take pleasure in killing a creature of God, especially a human being. The liar is asked to leave this land for good and never to come to it again.’

‘Is an offender never forgiven?’

‘Yes, if that person repents sincerely.’

‘Are you not allowed to see any man, except your own relations?’

‘No one except sacred relations.’

‘Our circle of sacred relations is very limited; even first cousins are not sacred.’

‘But ours is very large; a distant cousin is as sacred as a brother.’

‘That is very good. I see purity itself reigns over your land. I should like to see the good Queen, who is so sagacious and far-sighted and who has made all these rules.’

‘All right,’ said Sister Sara.

Then she screwed a couple of seats onto a square piece of plank. To this plank she attached two smooth and well-polished balls. When I asked her what the balls were for, she said they were hydrogen balls and they were used to overcome the force of gravity. The balls were of different capacities to be used according to the different weights desired to be overcome. She then fastened to the air-car two wing-like blades, which, she said, were worked by electricity. After we were comfortably seated she touched a knob and the blades began to whirl, moving faster and faster every moment. At first we were raised to the height of about six or seven feet and then off we flew. And before I could realize that we had commenced moving, we reached the garden of the Queen.

My friend lowered the air-car by reversing the action of the machine, and when the car touched the ground the machine was stopped and we got out.

I had seen from the air-car the Queen walking on a garden path with her little daughter (who was four years old) and her maids of honour.

‘Halloo! You here!’ cried the Queen addressing Sister Sara. I was introduced to Her Royal Highness and was received by her cordially without any ceremony.

I was very much delighted to make her acquaintance. In the course of the conversation I had with her, the Queen told me that she had no objection to permitting her subjects to trade with other countries. ‘But,’ she continued, ‘no trade was possible with countries where the women were kept in the zenanas and so unable to come and trade with us. Men, we find, are rather of lower morals and so we do not like dealing with them. We do not covet other people’s land, we do not fight for a piece of diamond though it may be a thousand-fold brighter than the Koh-i-Noor, nor do we grudge a ruler his Peacock Throne. We dive deep into the ocean of knowledge and try to find out the precious gems, which nature has kept in store for us. We enjoy nature’s gifts as much as we can.’

After taking leave of the Queen, I visited the famous universities, and was shown some of their manufactories, laboratories and observatories.

After visiting the above places of interest we got again into the air-car, but as soon as it began moving, I somehow slipped down and the fall startled me out of my dream. And on opening my eyes, I found myself in my own bedroom still lounging in the easy-chair!

Indian language NLP/NLU

Posted on by Anand Philip

I have been learning things about semantic search and somehow found myself back to wondering about how to train a Hindi or Malayalam language model. I have done this in the LSTM days, and while I’ve kept up with some research, I haven’t with the engineering. Anyway, so if you want to do some language modelling in indic languages, your best bet would be to begin with something that AI4Bharat team has come up with. This is a language modelling and NLP team at IIT Madras who have released what seem to be the latest and the largest models for Indic languages. Because of the sad  sad paucity of monolingual corpora in Indian languages, they have generally done multi lingual training.

 

To get an idea of just how bad things are,  these are how many tokens per language the largest models for indic languages have

Lets compare that with English, the C4 dataset based on commoncrawl is trained on 1.4Trillion tokens.

Several order of magnitudes larger

Even in terms of the variety of data, most English LLMs are trained on a mix of scientific articles, blogs, news, legal documents, books, novels etc. whereas AI4Bharat which collated the largest dataset is largely just news and Wikipedia.

Not only are the datasets small, the benchmarking tasks are also several order of magnitudes smaller. This means we can never really compare the performance of a indic model with an enlglish one.

Another problem : We don’t have good tokenizers for different Indian languages. The indicBert team uses the same bert tokenizer, designed for English,with some modification. Malayalam is an agglutinative language,it seems silly to represent it using a tokenizer made for enlish or hindi. There are several tokenizers who have tried to do this, but none seem to speak to larger models or with the transformers ecosystem and not too many people seem to be using them.

What all this means is that unless we really rally and get the datasets of our languages much larger, we are destined to have our languages represented exclusively with relation to English, which really is the approach google and others suggest for dealing with “low resource” languages. This is appalling.

I deeply appreciate the work the IITM team is doing, but someone needs to fund them and a dozen other labs like them several order of magnitudes more than they are.

 

 

citation for indicBert

@inproceedings{kakwani2020indicnlpsuite,
    title={{IndicNLPSuite: Monolingual Corpora, Evaluation Benchmarks and Pre-trained Multilingual Language Models for Indian Languages}},
    author={Divyanshu Kakwani and Anoop Kunchukuttan and Satish Golla and Gokul N.C. and Avik Bhattacharyya and Mitesh M. Khapra and Pratyush Kumar},
    year={2020},
    booktitle={Findings of EMNLP},
}

 

 

 

Ethical AI: Basic readings, schools of thought, an overview

Posted on by Anand Philip
Table Of Contents
show

Introduction

Over the last few years, everyone I speak to seems to know about AI. Ethical issues around AI are being actively discussed and debated, not just in the professional sense but also around coffee tables and at informal discussions by users.

This post is an attempt to provide an overview of the issues and approaches.

What we have now are essentially three interfaces of or approaches to ethics in AI.

Besides the differences in budgets, access to VC funding and who gets favorably written about in the NYT, 1 the main difference between the various factions (strange to see factions in ethics, but that is what we get for trying to pre-print our way into a science) is the temporal profile and the concreteness of the problems they are talking about.

Anyway my irritations aside, the ̷f̷a̷c̷t̷i̷o̷n̷s̷  approaches or interfaces are 2

  1. The professional ethics people
  2. The AI risk People
  3. The AI alignment n̸u̸t̸s̸ people

Professional AI ethics

The professional Ethics people are dealing with the immediate and current harms. They focus on identifying such harms and developing frameworks and knowledge that can be used to improve things now and for the future.  They are a lot like the bioethics people

One group of professional ethics people make guidelines. Another group fights big tech.Back then, when LLMs were not all that AI was, and people were using  regression models to predict recidivism, deciding who to hire, and identify people from video surveillance, these people were studying the harms of such systems and talking about what to do about it.3

Overview

Suresh, H., & Guttag, J. (2021). A framework for understanding sources of harm throughout the machine learning life cycle. In Equity and access in algorithms, mechanisms, and optimization (pp. 1-9).

This paper provides a model for understanding the harms and risks that arise in different parts of the model training and deployment process. This is a good high-level overview.4

Recidivism and algorithmic bias

  1. Algorithm is racist: ProPublica
  2. Algorithm’s feelings are complicated, results have a sensitivity/specificity tradeoff that is poorly studied: Washington Post

Linguistically encoded biases:

For many language models as well as image generation models King – Man + Woman = Queen and Man :: Computer Programmer as Woman :: Homemaker. These are looked into in the following papers

  1. Caliskan, A., Bryson, J. J., & Narayanan, A. (2017). Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334), 183-186. DOI:10.1126/science.aal4230 |Preprint on arxiv
  2. Manzini, T., Lim, Y. C., Tsvetkov, Y., & Black, A. W. (2019). Black is to criminal as caucasian is to police: Detecting and removing multiclass bias in word embeddings. arXiv preprint arXiv:1904.04047. (pdf)
  3. Malvina Nissim, Rik van Noord, and Rob van der Goot. 2020. Fair Is Better than Sensational: Man Is to Doctor as Woman Is to Doctor. Computational Linguistics, 46(2):487–497.

Pictorially encoded biases:

Facial recognition tech has a long history of being super duper racist, creepy, used for oppression, as well as not being very good. Tech companies, especially the superduperbig guys have been getting into this game and are releasing models that are seemingly better, but only if you’re a white male.

  1. Buolamwini, J., & Gebru, T. (2018, January). Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency (pp. 77-91). PMLR.
  2. White D, Dunn JD, Schmid AC, Kemp RI. Error Rates in Users of Automatic Face Recognition Software. PLoS One. 2015 Oct 14;10(10):e0139827. doi: 10.1371/journal.pone.0139827. PMID: 26465631; PMCID: PMC4605725.

LLMs and their problems

Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’21). Association for Computing Machinery, New York, NY, USA, 610–623.

Google the authors, and what they went through when they came out against LLMs. This will give you a lot of info about the stakeholders involved in this. The paper itself is highlighting the issues with Large Language models.

What about some practical stuff ?

A philosophical framework

While the debates go on, a zillion guidelines on how to deal with data and how to things ethically have come up. Chances are you will be overwhelmed if you start looking at them. I haven’t found any specific frameworks that speak to me. Personally I recommend and use the medial ethics framework along with the spider-man framework. which states that

cartoon image showing a clsoeup of someone's eyes, with captions on either side of the eyes reading "with great power" and "...comes great responsibility" this is from a spiderman comic

That is as self explanatory as an ethical dictum gets. The medical framework has the following four principles

  1. Respect for autonomy – Don’t make stuff that interferes with the autonomy of the user. When you’re in a position to make decisions for them, do this only after clearly explaining the harms to them and with their consent. Consent is king.
  2. Beneficence – an AI programmer should act in the best interest of the end user and not of the employer
  3. Justice – Is about the distribution of scarce resources, the decision of who gets what. Which means if your product, algorithm or system worsens disparities between people and creates inequalities, do better than that, think very hard about how you’re using your power.
  4. Non-maleficence5 – to not be the cause of harm. Also, to promote more good than harm, to the best of your ability. Also known as Above all, do no harm

Checklists and guidelines

If you’re looking for checklists and stuff that you can start using immediately to do responsible/ethical AI here is a github repo with the best links Awesome AI guidelines

If you want just *one* guideline, read this: Mitchell, Margaret, et al. “Model cards for model reporting.” Proceedings of the conference on fairness, accountability, and transparency. 2019. Model cards for model reporting.

This paper is great to get a conceptual understanding of the need for clear declarations about models. But it is too ambitious and a bit too bulky for wide use. Despite this, hugging face has implemented this on their website, although it isn’t always used. I recommend developing your own model card/checklist and having them attached to wherever you store your models.

This github repo with links to responsible AI resources is one more resource that focuses on practical advice and frameworks : Awesome Responsible AI

 Explainable AI

For me one foundational source of ethical issues with deep learning models is that the algorithm is a black-box.  The interpretability of the predictions or results of a neural network algorithm is poor. And so people who are working on model interpretability and explainability are also working on things that are critical for ethical decision making. 

AI Risk

Image of SWORDS military robot from wikimedia commons
The SWORDS system allows soldiers to fire small arms weapons by remote control from as far as over 3,937 feet (1,200 meters) away. This example is fitted with an M249 SAW.

 
The risk people are talking about existential and military and other big risks from AI. There is definitely some overlap between the risk people and the professional ethics people but a lot of what they discuss is about future or possible harms. Still very concrete harms and stuff that we can definitely see happening.

Think autonomous weaponized drones, robot soldiers,  autonomous robot doctors etc. The key here being to prevent things from getting out of hand when algorithms and robots are deployed to make autonomous decisions. I see this as  robocop-dredd-dystopia prevention work

Healthcare is another area where a great deal of harm could come from automation (a great deal of good too) and it is important that we think hard and work towards systems that safe-keep the interests of the patient.

This debate captures a great deal of nuance on  AI risk. Melanie Mitchell is a delight to listen to.

I don’t think there is enough work being done about real risk. We have a lot of thought leaders talking about it, but the engineering and the science of it is not getting a lot of traction.

AI Alignment

Imagine a world where an AI who is much smarter than us all has emerged and is currently demanding that everyone should call it lord and master and pray to it thrice a day.

The AI alignment people are

  1. Figuring out how to prevent such an AI from emerging and
  2. How best to align the interests of such an AI with our own.

I am not kidding you.

There are some actual cults involved and a lot of though experiments that are so bizarre I cannot even. 6

The problem is that these folks are completely ignoring addressing the  current harms using the doom and gloom of this possible outcome.

This is a doomsday cult kinda ideology. And sadly some very big names in AI have  signed up on this cult.

Worse, this approach is being used by some large players to superficially meet the demands of ethical AI, while completely sidestepping accountability for  the issues that are relevant today. As you can imagine, there are many important people in governments all over the world for whom the biggest worry is an AI that will replace them. So those guys are also treating this like its a real and credible threat right now.

There is no doubt that we need to ask ourselves this question, about how do we deal with systems that have more power than us. But I think the answer  to those questions lies in building in accountability, transparency, safety, informed consent and things that we already know how to do pretty well and don’t do because its bloody inconvenient and cost  money. I definitely believe that we need better engineering research into this, threat assessments, all that. But this issue is not so novel that we need to come up with an entirely new discipline which ignores and laughs at the stuff other experts on harm-reduction are saying. That is stupid.

I am not going to link to any of the alignment cultists but here are two  analytical articles about them which I think are great, and they link to plenty of stuff that you can explore.

Leopold Aschenbrenner: Nobody’s on the ball on AGI alignment ( this person works on an alignment team for one of the largest players ) 

Alexey Guzey :  AI Alignment Is Turning from Alchemy Into Chemistry

I dont really fully agree with these authors, but they make some sense and what would be an ethical guide without stuff that one disagrees with.

Concluding remarks

I admire greatly the activists who are fighting the good fight against Big-LLM, I really do. But I do not like agendas for change and progress being exclusively drafted by activists on social media. Social media is like reality TV, what you get if you do all your intellectual debate there is some form of Donald Trump.

I think that at some point the IT/AI engineering profession is going to realize the same thing that the medical world did. That if you don’t start doing things ethically, you will lose your power and create harms far beyond you can imagine and this shit will haunt you. If the tech world looks at the medical, i guess it can see a lot of unethical stuff. That is our shame.  But at the same time, I hope they will investigate the issue historically, and ask just how many checks and balances are there in healthcare to ensure the patient is not harmed. There is a lot to learn from the history of medical ethics.

Ethics make sense because it improve systems. Great AI will be ethical just like the best healthcare is ethical. And just like a doctor ultimately works with the patients best interest at heart, at some point AI engineers too  will adopt this dogma because it is the rational best choice and has a proven track record for reducing harm, and no one wants to build stuff that harms people.  Also, the workers of the world have a lot more in common with each other than with the bossmen. 7

This little fella knows ?she looks FABULOUS

Image from: Welch S. C. & Metropolitan Museum of Art (New York N.Y.). (1985). India : art and culture 1300-1900. Metropolitan Museum of Art : Holt Rinehart and Winston.

If you wish to cite this post here is the citation

Philip, A. (2023, July 11). Ethical AI: Basic reading, schools of thought, an overview. Anand Philip’s blog. https://anandphilip.com/ethical-ai-basic-reading-schools-of-thought-an-overview/

Ok that is all I have for you now folks, please do comment and subscribe and like and share this on boobtube instamart dreads and feathers as you wish.

Footnotes

  1. i.e. what kind of power does who have ↩︎
  2. Some people are calling this a schism, but it is not a schism ↩︎
  3. This work is then being carried forward by the actually-LLMs-are-not-that-great activists. ↩︎
  4. It however lacks the liberal-progressive-activism priorities ↩︎
  5. No relation with Angelina Jolie, having horns or dressing in black ↩︎
  6. I love thought experiments, they teach us a lot of things including that one must not confuse a thought experiment about a distant and remote possibility with something real and applicable now. ↩︎
  7. I am a Bourgeoisie malayali, can you blame me for bringing up Marx?. ↩︎

Surviving Violence: A study from Tamil Nadu

Posted on by Anand Philip

@swarraj’s tweet lead me to this report

 

 Source: Swarna Rajagopalan with ACR Sudaroli, Sandhya Srinivasan and Anu Aroon, Stuck in a Circle: Surviving Domestic Violence and Everyday Resilience in Tamil Nadu, Queen Mary University of London, 2023 [PDF Link]

This is my summary and some takeaways

Technical Details: This is a multi-method study which collected primary data from survivors of domestic violence and other key stakeholders,  centering the experience of victim-survivors. It was conducted in 3 districts in Tamil Nadu; Ariyalur, Chennai and Vellore. The sampling method was Snowballing. 60 woman and 1 transman survivors were interviewed. Besides them, witnesses, members of the society and family were interviewed, as well as 62 support-providers like lawyers, medical workers,  NGOs, and the Police.

There are seven sections in the report.

Section 1 SETTING THE SCENE

The report opens with background information about the state of Tamil Nadu from key studies based on  NFHS, SDG, Census etc. highlighting the data on women’s health and experience of violence, and goes on to describe the questions in the interviews and the responses and then describes the methodology in detail.

Section 2  UNDERSTANDING DOMESTIC VIOLENCE

It then goes on to describe attitudes and beliefs of the interviewees about gender violence, laws about it, what constitutes it, and stances on punishment and resolution etc.  are elicited.

The interviews were done with great sensitivity and did not shy away from asking very uncomfortable questions. There were both structured as well as non-structured bits in each interview. Some of the questions were

  • Why do you think women experience Domestic Violence?
  • If a friend or family member told you they were experiencing domestic violence, what advice would you give them?
  • Is there ever a situation/circumstance when a woman should stay in an abusive/violent relationship? If so, why?

Some of the findings from attitudes towards violence are

  • Survivors understood that beyond physical violence, sexual, emotional, verbal and economic abuse constituted domestic violence.
  • They agreed that women who experience violence should seek help.
  • Community members agreed with these views, and  that domestic violence is committed because of cultural factors, interpersonal misunderstandings and alcohol abuse.
  • Community members also believed that women should and would put up with the violence up to a certain threshold of cruelty or affecting children, before leaving.
  • Support service providers largely agreed with an inclusive definition of domestic violence, but were less sure about whether it was a criminal office and the perpetrator should be punished

Summarized in the report as:

While society’s understanding of domestic violence has broadened, the a significant percentage believe that it is to be expected in relationships and are ambivalent about how best to respond. A survivor therefore steps out of an abusive situation into a support ecosystem that may or may not validate her experience.

Section 3 THE EXPERIENCE OF VIOLENCE  looks into the details of the experience of violence (who, how, what etc.), why survivors think they are subjected to violence, ways of coping, seeking help etc. The community is also asked the same questions but keeping the experience of the survivor as the center.

Section 4  looks into the support  available to survivors. This includes friends, family, the police, NGOs, lawyers etc. and their experiences in dealing with domestic violence and their understanding of the experience of it, the causes of it and the ways to solve it.

Section 5  looks into how well the Protection of Women from Domestic Violence Act, 2005 is implemented  (I didn’t read this too closely)

Section 6 asks what patriarchy looks like, with stories from survivors and witnesses and others. eg

Section 7 deals with the people’s views on developing resilience and what it means. This section spoke to me deeply.

This project arose out of a broader quest to understand resilience in both intellectual and practical terms. Where do humans find their resilience? What are the external supports and impediments for it? How does any society (or state) build the resilience of its people in the face of adversity? We took this question to a section of people vulnerable because of their gender in a patriarchal society and because of their everyday experience of violence in the very contexts that are meant to keep us safe—intimate relationships, family and home

The report ends with suggestions to what both the civil society organizations and the government apparatus could do to improve things, informed by study.

Each of these suggestions is a gem and carefully argued. Everyone should read the report [PDF Link].

Studies  from other  states in India and other policy briefs can be found in the resources section of the project’s website

Takeaways:

  1. At a cultural level, the understanding of domestic violence has changed a lot, and while everyone seems to agree that it is a bad thing and ought to stop, the solutions are not all that clear or trustworthy (to the society).
  2.  We need to  support survivors of domestic violence in the communities we are part of. The support can be in many ways, but it should probably begin with speaking up  about this in our families and providing support to family members who are survivors. I think this is key. We need to be clear and loud as individuals about what our stance is in the family-social units we are part of.
  3. Individuals should attempt to provide structural support in some forms more widely.
  4. Is there a model of heterosexual marriage that is not based on privileging the male and asking for submission from the female? Without widespread demonstration of the alternate model, for the family unit and the greater culture that we are part of, there really is no alternative that can be adopted.
  5. Women who leave abusive spouses and/or seek help and  justice need to be celebrated.
  6. There is really no benefit to women “adjusting”. Adjusting is based towards benefits the norms that created this problem. I recognizing that that is the only recourse available in many situations at present.
  7. Corruption masquerades as pragmatism.

I could view this problem as legacy code issues. While the number of bugs in the system make you want to burn it all down or lose hope, most of what we have came via iterative development. I mean the methods, techniques, heuristics, design-patterns and more that make up culture. that is reductive, i agree but useful to me.

The report makes me grateful to the feminists who have fought to get the laws and legal structures in place to deal with this issues.

It makes me hopeful to read the vast majority of the interviewees want education, financial stability, and social support and see these as being part of the cure.

This is reinforces my  belief that individual financial empowerment is severely understated and underutilized as a source for reducing harms in the nation at large.

What level is your EMR/EHR?

Posted on by Anand Philip
Level 1 EMRs

Warning: The following text is needlessly cruel

Level 1 the PITA : EMRs that double the work — systems where the EMR is used for documentation or compliance, but all the real work happens in written form, so this just increases the net amount of work done. [Admittedly, this is a problem of the administrative system, not so much the EMR itself, at times] (a large  majority of  EMR deployments in india are this kind)

Level 2 the copycat : EMRs that successfully replicate all the physical record keeping and so work isn’t double, but  worsens things because replicating physical workflows in-toto  invariably will make everything slower (This is what most new EMR implementations are)

Level 3 The trying-very-hard-to-be-cool: EMRs that have some automation and short cuts built in, like medication templates, discharge summary templates and suchlike which in some areas, significantly reduce the time taken to do a task and are overall nearly as fast as paper and pen ( I’ve seen a few of these)

Level 4 The we-do-design-thinking-and-listen-to-the-money : EMRs that have thought through the clinical processes  and removed all parts that do not need to be there, and have largely click and pick workflows these are faster to use than other EMR/EHR and might even be faster than pen and paper. (I’ve seen maybe one of these of these)

Level 5 : EMRs that are faster than pen and paper, assist and improve decision making and make things easier for patients. (Haven’t met one of these so far)

Book review: The Truth Pill by Thakur & Thikkavarapu

Posted on by Anand Philip
Table Of Contents
show

An Important book written in ALL CAPS

Topics covered in this book:

  • How approval for new drugs, new fixed drug combinations and new generics are supposed to work and how they actually work
  • What are the quality standards and monitoring mechanisms in place for scientific medicine, as well as AYUSH.
  • Laws governing AYUSH, scientific medicine and the production of drugs in India.

What I liked about the book:

  • Thorough research into how things are supposed to work in the area of drug regulations in India and an even more detailed research and experience into the million ways in which it is failing to work. Seriously, these two know their drug regulation problems.
  • Thorough understanding of the main players, organizations, lobbies etc. involved in this messy situation
  • Stupendous amounts of cred as far as I am concerned due to being a whistleblower as well as a RTI crusader/activist and all the litigation they have tried.
  • Focus on transparency, access to information, rights to quality
  • Documentation of various types of issues and the corruption in different bodies
  • Clear-sighted tracking of the bureaucratic games and political shenanigans around drug laws in the country.

What I learned from the book

  • We have a lot of outdated laws regulating drugs and other allied health products. These are exploited and used by both the governance people and the drug companies to push a lot of outright shitty medicines to the public
  • There is a LOT of corruption, incompetence and lack of transparency in the organizations that are supposed to be watchdogs and regulators
  • Successive governments and bureaucracies have generally found ways to pass the buck, do ass-saving paper pushing and pandering to sentiments rather than make any substantial and scientific changes.
  • We have a lot of under-equipped and underfunded drug regulators and inspectors
  • The quality of drugs produced in India is for all practical purposes at the mercy of the individual pharma company that does the production and there’s very little if any, real oversight or really punishment for even serious lapses.
  • Every month somewhere in this country, at least one pharma company is shown to have produced sub standard drugs and even then the judiciary seems to think harsh punishments are not needed. I mean, people have died and the dudes who made those meds never saw jail time.

Problems with the book

The whole book is narrated by someone who is shouting at YOU AND ALL THE IDIOTS WHO RUN THIS COUNTRY AND ALL THE CORRUPT DRUG MANUFACTURERS, THE INCOMPETENT JUDICIARY, THE CLUELESS LAW MAKERS OF NEWLY-INDEPENDENT INDIA WHO DID NOT HAVE 21ST CENTURY VALUE SYSTEMS OR KNOWLEDGE AND DRUG RIGHTS ACTIVISTS WHO ARE ALL CORRUPT TOO.

My blood pressure was measurably higher when I was reading this book. Seriously. There’s like four people the authors have anything nice to say about from 1923 -2023 (and two of them are dead). Oh, and the USFDA. They LOVE the FDA and the way things are done in the United States. 

<sarcasm> I mean, I don’t blame them, they’ve been fighting the good fight all their lives when literally everyone else in India is asleep, incompetent and corrupt. </sarcasm>

I get it, though, one of the authors has been constantly targeted and harassed by various government agencies and big pharma. So I get where the tone is coming from, I know righteous indignation, I used to have a lot of it. Now I just have reflux and indigestion.

But the roid-rage-hulk-smash tone of the book made me want to put it down and stop reading about fifteen times. I stayed the course only because they did thorough research, and what is an activist without some righteous rage, really. 

The other issues are that the policy and legal recommendations made in the book without fail made me wince, flinch and facepalm. The authors forget just how poor this country is, and just how many people there are.  The authors also don’t know a damn thing about the practice of medicine in the real India. They are operating from an america-centric and america-learned model of how health should be dealt with.

The most important implied recommendation of this book seems to be BURN IT ALL DOWN. 

Throughout the book they mention the dozens of parliamentary and other committees that have looked into these problems, and how so few of these reports see the light of day and fewer still get anything changed. Because the suggestions are not useful, or due to the unwillingness of the governance system to make any real changes.

This book has the same problem.

For eg.

  • They repeatedly suggest stronger laws, after describing just how poorly existing laws are implemented.
  • They suggest that online pharmacies will solve the supply chain problem (omg i cannot even)
  • They suggest that small pharmacists who are found to have bad drugs should be punished heavily.
  • They repeatedly recommend increasing the legal pressure on smaller players in the supply chain.

You cant improve state capacity by making tougher laws.

You can’t fix corruption in high places by punishing the small guy.

Some medication is most definitely in many many cases better than no medication, talk to a doctor, any doctor.

You cannot copy-paste quality standards designed in USA and Europe to solve for India.

That’s not how anything works. 

Would I prefer my patients got 40mg of  Telmisartan every time they buy a drug, YES.

Do we need to improve the capacity of drug inspectors and dis-entangle the regulatory nightmare, HELL YES.

But do we do it by shutting down everyone who only has 20 mg of telmisartan in their 40mg tablets, heavens no. The 20 mg the patient gets at 0.05 Paise a tablet is worlds better than the 0 mg he will get at 2 rupees a tablet.

In the words of the y00ths I interact with, the authors desperately need to touch grass.