We are not building tools anymore. We are building decision-makers — systems that will shape who gets a loan, who gets a job, whose identity is trusted, and whose is erased. The ethical architecture of those systems is the most consequential design problem of our generation.
I. The Question of Intelligence
We call it Artificial Intelligence. The word is doing enormous work. Intelligence implies understanding — the capacity to weigh consequences, to feel the weight of a wrong decision, to wonder whether what you are about to do will matter in ten years. A machine does none of these things. It finds patterns in data with extraordinary speed and scale. That is a profound capability. It is not wisdom.
The conflation matters because it creates a dangerous transfer of moral authority. When we call a system intelligent, we unconsciously begin to trust its outputs the way we trust an expert. We defer. We stop questioning. And in that deference lies the seed of the problem this article is about.
The machine has no stake in being right. It has no family to protect. It will not be there in 2045 to witness the consequences of the decisions it helped make in 2025. The people who build it will. That asymmetry is not a technical detail — it is the central ethical fact of our moment.
The machine has no stake in the outcome. It will not be there in 2045 to witness the consequences of the decisions it helped make in 2025. The people who build it will.
II. The Feedback Loop That Should Terrify Us
Here is the mechanism that should concern every engineer, policymaker, and citizen: AI systems learn from human input, and humans gradually learn to give input that the AI rewards. This is not science fiction. It is already happening.
When social media recommendation engines rewarded outrage, the media ecosystem produced more outrage — not because journalists became dishonest, but because the feedback loop made emotional content feel like success. The algorithm didn’t lie. It simply made truth slightly less rewarding than performance, and humans adjusted accordingly.
Scale that dynamic to the systems governing civic identity, credit access, health triage, or criminal sentencing — and the stakes are no longer about clicks. They are about whose life is reduced to a risk score, and whether any human being is left in the loop with the power and the incentive to say: this is wrong.
The loop closes like this: a model learns from data → humans shape data to please the model → the model’s biases harden → the humans serving that model stop noticing they’ve been reshaped. By the time anyone looks up, the ethical ground has shifted beneath them and the original intent is a historical footnote.
III. The DigiLocker Problem — A Case Study in Systemic Vulnerability
⚠ Critical Vulnerability — Identity Infrastructure
India’s DigiLocker and M-Parivahan represent a landmark achievement in digital governance — millions of citizens now carry legally valid documents on a smartphone. But the same infrastructure that makes life easier for honest citizens creates a new attack surface for bad actors equipped with AI.
The threat is specific: generative AI can now produce synthetic images of licences, identity cards, and permit documents that are visually indistinguishable from originals to a human eye. When a police officer on a highway verifies a document via QR code, they are not checking the document — they are checking whether the code resolves to a valid server record. If the QR code is cloned and the server record is spoofed convincingly enough, the human checkpoint becomes ceremonial.
This is not a theoretical future state. The component technologies — generative image synthesis, QR manipulation, server-side record injection — all exist and are accessible. What is lagging is the institutional architecture to detect and respond to their convergence.
The solution is not to abandon digital identity infrastructure — that would be a catastrophic regression. The solution is to build adversarial resilience into the system from the start: cryptographic signing at the document level, real-time anomaly detection, and independent red-team auditing that is not funded by the agencies being audited. Each of these requires both technical competence and institutional courage — which brings us to the chain of responsibility.
IV. The Chain of Responsibility — And Where It Breaks
01 The Builders — Engineers and Students
The first and most intimate link. Every architectural decision, every shortcut, every “we’ll fix the bias later” carries consequences that outlive the sprint. Ethical responsibility begins at the keyboard — not in the ethics committee meeting that happens six months after launch.
02 The Teachers — Institutions and Academia
Engineering curricula worldwide still treat ethics as a soft elective rather than a load-bearing pillar. Until a student cannot graduate without demonstrating they can identify and mitigate algorithmic harm, we are producing technically skilled people who are ethically unequipped for the systems they will build.
03 The Guides — Domain Experts and Ethicists
Technical experts who understand societal impact must be embedded in product teams — not brought in as consultants after the fact. A lawyer brought in after a contract is signed cannot un-sign it. An ethicist brought in after a model is deployed cannot un-deploy its biases into three million decisions.
04 The Testers — Independent Auditors
This is the link that most frequently snaps. Auditors hired and paid by the institutions they audit have structurally compromised independence. Meaningful oversight requires adversarial testing by parties with no financial stake in a clean result — and with legal protection to publish what they find.
05 The Guardians — Government and Regulatory Bodies
Governments that adopt AI systems must also fund the institutions that scrutinise them. A ministry that deploys an automated benefits system and simultaneously defunds the ombudsman that handles complaints has not implemented AI — it has implemented impunity at scale.
The chain is only as strong as its weakest link. In practice, links 04 and 05 are routinely absent, underfunded, or captured by the interests they are meant to check. That is not a technology problem. It is a political economy problem — and solving it requires exactly the kind of sustained public pressure that informed citizens can generate.
V. The Six Principles That Must Be Non-Negotiable
Fairness
No Algorithmic Caste System
Systems must be tested across demographic groups before deployment — not just on aggregate accuracy, but on differential error rates. Whose errors cost more?
Transparency
Explainability as a Right
Any person affected by an automated decision has the right to a plain-language explanation. “The model said so” is not an explanation — it is an abdication.
Privacy
Data Collected for One Purpose, Used for One Purpose
Health data cannot quietly become credit data. Student performance data cannot become employer profiles. Consent must be specific, not buried in terms of service.
Accountability
Named Humans at Every Consequential Node
Every AI-assisted decision with real-world consequences must have a human accountable for it — whose name, title, and contact are publicly recorded.
Security
Adversarial Robustness by Design
Systems must be red-teamed before deployment. Not after the first incident. The threat model must assume motivated, well-resourced adversaries — not honest users.
Humanity
The Override Must Always Exist
No system should make a decision from which there is no human appeal. The right to a human review is not a luxury feature — it is the boundary between a tool and a regime.
VI. What Builders Owe the Future
There is a specific kind of moral recklessness in building systems whose consequences you will not live long enough to witness. The engineer who cuts a corner on identity verification in 2025 will never meet the person whose life is upended by that corner in 2038. The academic who treats AI ethics as a theoretical seminar will never counsel the family whose loan was denied by a biased model trained on their own research.
This is not about blame. It is about the nature of the obligation that comes with technical power. To build a system that shapes human life is to take on a duty of care that does not expire when the project ships.
India’s technology ecosystem is in a formative moment. The decisions being made now — about what gets digitised, how identity infrastructure is secured, what oversight mechanisms are built alongside the systems rather than retrofitted after the scandals — will determine the ground rules for a billion people’s relationship with algorithmic power. That is not an abstraction. It is the shape of the next generation’s world.
The question is not whether AI will be powerful. It will. The question is whether the people building it will be honest about its limits, courageous enough to slow down when honesty requires it, and wise enough to remember that the purpose of technology is not to make machines smarter — it is to make human life better, fairer, and more dignified.
The purpose of technology is not to make machines smarter. It is to make human life better, fairer, and more dignified — for the people who will be here long after we are gone.
VII. A Note on Truthfulness
The author of the provocation that inspired this article ends with a simple word: truthfulness. It is the right word. Not accuracy — systems can be accurate on average and catastrophically wrong for specific people. Not safety — safety is a bar set by those in power, and the bar is often set where it is convenient. Truthfulness.
Truthfulness means building a system and being willing to say publicly what it cannot do, who it will harm, what assumptions it makes, and what happens when those assumptions are wrong. It means the courage to ship slower, to push back on timelines, to tell a client that the system is not ready, and to mean it.
That courage is not produced by regulation alone. It is produced by people who have been taught, from the beginning, that they are not just engineers. They are stewards. And the thing they are stewarding is not code. It is the future.
“Are you going to leave your family with an unsafe world? This will haunt them after you and I are long gone. Think about it.”
— The question every builder must answer before the next commit.