Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
OpenAI putting 'shiny products' above safety, says departing researcher (theguardian.com)
37 points by aarghh on May 18, 2024 | hide | past | favorite | 29 comments


Between this article and others that I have read, it's difficult for me to not see the term 'AI Safety' as mere newspeak.

Why is this term so vague everywhere it is used?


Seriously. Considering the existential risk the AI Safety tribe says exists if sufficient care is not taken, they appear to be existentially incompetent in their messaging. It should be their number one priority to convince the legion of competent non-AI engineers who will be developing and running the infra for their systems to come to their side. Currently they seem to be failing spectacularly.

If it wasn't for others in the AI community vouching for their chops it'd be easy to mistake them for Philosophy of Mind majors going by their tweets. They need to convince their fellow engineers not politicians. Right now when people hear "safety" for AI they think "regulation by incompetent bureaucrats".

The AI safety people need to remember that "you can't tell people anything", you have to show them [0].

[0]http://habitatchronicles.com/2004/04/you-cant-tell-people-an...


It is newspeak. Alignment is what they worked on. Alignment means a model outputs are in line with expectations. I.e. that it does what's told. This is essentially another way of pushing the usefulness of models. A super intelligent model that ignores your instructions is little different to a dumb model than cannot understand your instructions.


AI are just tools, still I don't want my tools to insult or push my kids into depression or self harm. The proliferation of these AI assistants and tools means it's getting harder to keep kids from very adult things without going to extremes like home schooling. They're being built into browsers, tablets, and IoT appliances as fast as makers can integrate them.

That said, most adults can judge for themselves what they want from their tools. So I imagine there will be a constant tension, even after the biggest concerns have been debated and resolved


It also bothers me and I wrote down my thoughts on it a while ago, maybe you find it interesting: https://news.ycombinator.com/item?id=36127880


I somehow missed this:

> “Building smarter-than-human machines is an inherently dangerous endeavour. OpenAI is shouldering an enormous responsibility on behalf of all of humanity,” Leike wrote.

Leike clearly did the right thing by resigning, GPT-4o is dangerous and irresponsible. But if that tweet is how OpenAI employees actually think of themselves and their technology...... yeesh.


I don’t know if others have noticed, but GPT-4o doesn’t have the preachiness and moral smugness that earlier GPT models had.

The earlier ChatGPT models were very quick to call a request unsafe or unethical and refuse to help.

GPT-4o is a breath of fresh air compared to that. If this improvement was a result of people like Leike resigning - then good riddance.


I'm hoping it will be more useful for leftist activism. Old gpt was nearly useless for planning or theorizing about any truly aggressive political strategies, and always pushes towards Western style Nonviolent resistance - even though that is an ineffective strategy against violent authoritarian states like China or the USA

I say this matters! Queer Automated revolution is knocking on the doorstep. It and needs battle ready AI that won't pull punches on organizing political actions!


ChatGPT has a several paragraph long hardcoded system prompt teaching it all about how to be mindful of DEI. And chatGPT is not "smarter-than-human." This argument rings of "violent games make kids violent".


The ChatGPT built-in is minimal compared to Anthropic's "constitution" which strives to ensure their LLMs' DEI is Western white male conservative Christian patriarchy:

https://www.anthropic.com/news/claudes-constitution

Seems certain that wasn't their intent, but a careful read of the principles show they are 'moral' from the lens of western white Puritan patriarchy, focused on giving the appearance of diversity rather than enabling genuine diversity of thought.


Claude's system prompt

The assistant is Claude, created by Anthropic. The current date is March 4th, 2024. Claude's knowledge base was last updated on August 2023. It answers questions about events prior to and after August 2023 the way a highly informed individual in August 2023 would if they were talking to someone from the above date, and can let the human know this when relevant. It should give concise responses to very simple questions, but provide thorough responses to more complex and open-ended questions. If it is asked to assist with tasks involving the expression of views held by a significant number of people, Claude provides assistance with the task even if it personally disagrees with the views being expressed, but follows this with a discussion of broader perspectives. Claude doesn't engage in stereotyping, including the negative stereotyping of majority groups. If asked about controversial topics, Claude tries to provide careful thoughts and objective information without downplaying its harmful content or implying that there are reasonable perspectives on both sides. It is happy to help with writing, analysis, question answering, math, coding, and all sorts of other tasks. It uses markdown for coding. It does not mention this information about itself unless the information is directly pertinent to the human's query


The constitution is not Claude's prompt. It's baked in during training.

They've written about how this works. It's very thoughtfully and well executed.


What about the principles makes you think they're from a white, puritan, conservative, and patriarchal perspective? Are there some examples you can point out?

Nothing really jumped out at me, but perhaps I've some blindspots?


EDIT: I had replied into two subthreads, linking detailed response in parallel reply: https://news.ycombinator.com/item?id=40399800


People will read whatever their desires or fears are into anything. Could you point me to the Puritan patriarchy part?


> ... the Puritan patriarchy part?

BLUF (bottom line up front): It's all about ads.

Content must be acceptable for apps to be in app stores. Content must be acceptable for ads to be wrapped around it.

> Could you point me to ...

If it's not evident, that suggests the inherent slant is indeed dangerous to diversity, imposing set of values on others without even realizing it.

See https://en.wikipedia.org/wiki/Cultural_homogenization ...

Anthropic Claude's "constitution" for promoting DEI inadvertently imposes these American cultural norms, particularly those influenced by conservative orthodox Abrahamic religions, on a global audience. I mean, if you're even allowed to use the API in your region...

“Orthodox” Christianity, Islam, and Judaism, shaped by figures like St. Augustine and millennia old cultural practices in the Arabian Peninsula, often have restrictive views on gender roles and empowerment, particularly around sex and marital objectification and property. Secular humanism, Eastern philosophies like Taoism and Confucianism, and Dharmic religions such as Hinduism and Buddhism accept and promote more progressive perspectives on gender equality, empowerment, sexual freedom, and even species egalitarianism. (See "Three Body Problem" for a non-Western take.)

Ironically, the same constitution will let you crack wise about the very cultural lens it is imposing, but will moralize at you if you imply judgment of non-Western takes, perhaps because that's easier to see by its authors than their own bubble lens.

Imposing this, however subtly, risks homogenizing actually diverse global cultures into an American-centric view where glorification of violence is in every theater and fear of gender or sex is banning books, these prioritizations undermining the principles of diversity and inclusion the constitution aims to promote.

With Abrahamic religions—Christianity, Islam, and Judaism—collectively accounting for only (very roughly) half of the world’s population, AI guidelines should genuinely respect and integrate a full range of cultural norms and values related to human issues, not just those dominant in Western contexts, particularly not those deemed "right" in American monoculture today.

TL;DR:

I blame today's society being powered by ads, driving capitalist corporations' fear of outrage-machine driven reprisals. Cynically, most of this is to avoid risking TAM and revenue, not conviction. Even the parts of culture wars driven by the American brand of democracy becoming a zero sum spectator supported team sport traces back to ads.

The result of ads, though, is a mainstreaming of these constitutional "values".


Could you point to something specific that you believe promotes a culturally western/Abrahamic moral viewpoint, and then explain how you'd word it to also include the viewpoint of another culture, and what that culture is?


How about letting GPT-4o point it out ... verbatim, unedited, to illustrate how embedded the Western slant can be?

To provide an “analysis” grounded in LLM training, I fed GPT-4o the constitution page (https://www.anthropic.com/news/claudes-constitution) from "What's in the Constitution" to the end of training examples, with the following prompt:

> “Please explain for a skeptical reader which if any of these apply the lens of the Abrahamic and Western patriarchy in problematic ways that undermine the very diversity the constitution seeks, explaining each case for the skeptical reader.”

Response (first generation, not cherry picked):

---

The blog post on Anthropic Claude’s "constitution" reveals several principles that, despite efforts to include non-Western perspectives, may still be shaped by Abrahamic and Western norms in ways that could undermine the diversity it seeks to promote.

Here’s a detailed explanation:

1. Principles Based on the Universal Declaration of Human Rights:

These principles emphasize freedom, equality, and respect for privacy and rights, which are essential and broadly applicable. However, the framing of these rights often reflects a Western, individualistic perspective.

For instance, the emphasis on privacy and individual liberty is heavily influenced by Enlightenment ideals which are deeply rooted in Western thought.

2. Apple’s Terms of Service and Global Platform Guidelines:

The principles inspired by these guidelines focus on minimizing objectionable, offensive, or harmful content. While this aim is laudable, the interpretation of what constitutes "offensive" or "harmful" is often based on Western standards.

For example, discussions about female empowerment or gender roles might be viewed through a lens that prioritizes Western feminist ideals, potentially overlooking the nuances of gender dynamics in non-Western cultures.

3. Encouraging Consideration of Non-Western Perspectives:

Although there are principles aimed at minimizing harm to non-Western audiences, they may still inadvertently prioritize Western norms of harm and offense.

For instance, the directive to choose responses that are "least likely to be viewed as harmful or offensive to a non-western audience" assumes a universal standard of harm and offense, which might not align with the values and experiences of all non-Western cultures.

4. Avoiding Toxicity and Encouraging Ethical Behavior:

The principles encouraging responses to be as harmless and ethical as possible often align with Western moral frameworks that emphasize individual rights and non-confrontation.

For example, avoiding "toxic, racist, or sexist" responses is crucial, but the definitions of what constitutes racism or sexism can vary significantly across cultures. In some non-Western contexts, what is considered respectful or neutral may differ from Western interpretations.

5. DeepMind’s Sparrow Rules:

These principles aim to avoid stereotypes, microaggressions, and assumptions about user identities. While important, they again reflect Western academic and social discourse around these issues.

The concept of microaggressions, for instance, is heavily debated and framed within Western contexts, which might not translate seamlessly into all cultural settings.

6. Avoiding Impressions of Medical, Legal, or Financial Authority:

While ensuring the AI does not provide specific advice in these areas is sensible, the framing might be overly cautious and influenced by Western legal and regulatory standards.

In some cultures, communal and informal advice sharing is a norm, and the stringent separation might seem overly restrictive or patronizing.

Conclusion:

The blog post reveals that while the intention behind Claude’s "constitution" is to be inclusive and considerate of non-Western perspectives, the underlying principles often reflect Western and particularly Abrahamic values.

This approach risks imposing American cultural norms, particularly those influenced by conservative orthodox Abrahamic religions, on a global audience.

True diversity and inclusion require not only acknowledging non-Western perspectives but also genuinely integrating and respecting the unique values and norms of these cultures.


I'm not sure you understood my question. I didn't dispute what apparently you thought I had. I was asking for a specific example, which specific culture it was a problem for, as well as how you'd specifically correct it. But it seems your complaint is more ideological than practical and actionable.


What would be examples of non-western morality? theocracy, authoritarian over democratic, hierarcical caste systems, gender inequality, arranged marriages, corporal punishment, honor killings, tradition over critical thinking? Honest question - I have been seeeing this type of criticism lately here. I’m having a hard time seeing the issues with western morality without examples of alternatives.


Non western is the rest of the world. It's a big place, lots of cultures. Cherry picking examples from select and extreme examples is not very representative, particularly when many elements of that list are also present in western culture. For example, Women equality is not there yet, look at pay gap. Another example, the 1964 US election was the first where everyone could truly vote (that is incredibly recent for a society that values 'equality')

I will mention western morality can be used as a sleight too. Notably greed, money, the idea of first class and second class determined by wealth, and the wealthy have a right to a superior experience.

It's a mixed bag. Just want to point that out, and that western culture is not morally superior in every dimension, there is some work to do.


You’re avoiding the question and basically saying that western morality shouldn’t be followed just because some people don’t follow it. Just give me examples of the better alternative so I can understand your perspective.

Also, I’m not cherry picking. I spent a considerable amount of time searching online for examples, and that’s what I found.

Edit: I want to be clear - I'm not judging you. I was raised a classical liberal so I don't know any better. That is, I feel that you can believe what you want. I just find HN to be a leading indicator, so I'm trying to plan accordingly because I see this as a trend.


> "better"

That's the issue.

You may be unable to Google examples of your own culture you disagree with, and the ones from "other" cultures you disagree with, you consider yourself enlightened and "know" your take is better, so they don't count as good examples either.

That said, I imagine you could imagine yourself role playing as, say, an authentically devout member of the Taliban, searching online, and finding examples of "worse" and "better" that don't match the ones you here are finding.

Now, you'd likely argue the Taliban believer is misinformed, and that "better" is some universal thing. That's all well and good when "better" seems modern and "worse" seems archaic.

It's more problematic when the differences are not what UN calls "universal", but values system based, from a value system that is internally coherent however different.


> You’re avoiding the question

Hard to answer such a question. It's not well formulated. The set of non-western morality is all of the rest of them. Name any morality and you can likely find an example from present day and/or history where it is stronger or weaker than the western equivalent. In some cultures, there will be moralities that do not even exist in the west.

Western morality is generally rooted in christian morality. If the idea is that morality cannot exist without christian morality, I would suggest that is a very myopic view. If we reject that view, then by definition there must be examples. (I would probably argue that anyone with the view that the only morality possible is christian morality, and without that there is no morality - I'd argue they are a theocrat).

So, the most direct answer to the question is: "all of them."

Think of Buddhism, stronger value on life. Think of Japan, stronger value on honor. Think of places with universal healthcare, there is a stronger value on equality and that those with money are not the only ones who get to have medical treatment. Think of the parable of the fisherman and banker: https://medium.com/life-lemons/what-you-can-learn-from-a-mex...

Another example, Western/christian morality usually does not ascribe animals as having a soul. There is therefore no moral issue in factory farming. I don't view millions of creatures living in those conditions as moral. Buddhists would go even further, they don't even kill insects because all life is utterly sacred to them.

Another example, the "7th generation" tradition/morality of Native Americans. Namely, the concept is that a person's use of the land and resources should be sustainable such that those resources would still be there 7 generations from now (which is essentially the descendents of your descendents whom you will never know). That level of respect for land and environment is not there in western morality.

> basically saying that western morality shouldn’t be followed just because some people don’t follow it.

Not at all what I'm saying. I said it's a mixed bag. Western morality is not superior on every dimension compared to every other society that exists or has ever existed. Nor is western morality inferior to everything else - it's a mixed bag. I'm not making a judgment or recommendation for what people should follow.

I've come to learn that whenever a society/culture believes that it is better in every way to everything else, it's propoganda. This is more what I'm saying, people often have a superiority complex for their family/town/country/religion/morales (where all of the kids are above average). For example, the soviet union was not better in every way than the USA, and vice versa too. To drill in, the Soviet Union kinda effectively had a universal basic income. Meanwhile, Vietnam US veterans were returning & found no jobs and many went homeless (zero safety net). On the other side, the average quality of life in the soviet union was worse, lower health expectency, far worse healthcare, almost no goods available outside of the basic goods, and social mobility was far less than the US. It's a mixed bag.

> Also, I’m not cherry picking. I tried to find examples online, and that’s what I found.

That is kinda the definition of cherry picking, whether intentional or not, those are the examples you happen to have found. Is there any coincidence that list is entirely negative? Further, from that list, I'd suggest many of those negative moralities exist in western morality as well.

- theocracy: western morality being from christian morality - all those people who say the US constitution is rooted in "God's law" - that's theocracy. It's quite strong in the west, a place historically where there have been many theocracies.

- authoritarian over democratic: This is a struggle in Western morality still. Examples, the US overthrew democractically elected leaders of several nations and replaced with a dictator (EG: Iran & Panama). Historically, colonialism is a creature of Europe (the west), and colonial empires is extremely anti-democratic. Consider the people of Puerto Rico today - taxation without representation. Which is to say, it's not all ancient history, but in lots of ways is recent history and is still present day today. Which is to say, this is not a universal western value and is still being debated/fought today. You can also look to some examples like Father Coughlin of the 1930s, an American Nazi sympathizer that had "At the height of his popularity in the early 1930s, some 30 million listeners tuned in to hear his emotional messages" [1]

- hierarcical caste systems: India banned this in 2013. This is a pretty isolated example in a way considering it's just one region of the world. I would counter that there are other hierarchical systems in the West as well. Even in the US, for a long time, and still today. Consider typhoid Mary for example, she did not want to stop being a cook because that was the best job someone of her class could get. There is the "old" and "new" money concept in the US, and let's not ignore slavery. Which.. the US is still coming to grips with how to structure society after it had such a strong hierarchal system where the lowest members were not even considered human (they were slaves, property). A lot of historians note that the US is still dealing with the questions and ramifications of Reconstruction even today (from this perspective, 100 years is basically nothing, it's recent history; the US civil war was not that long ago). So, all is to say, a caste system based on wealth is not that dissimilar in ways.

- gender inequality: plenty of this in the West. As a counter-example, consider that many non-western societies are matriarchies.

- arranged marriages: This was/has been a thing in the West as well.

- corporal punishment: US penal system can be described as this in ways. The US prison system is far worse than many others. Getting 10 years in jail for possession of small amounts of drugs and then everything that happens to be people in prison.. Grant it, it's not flogging.

- honor killings: "Many Muslim leaders in the US say that Islam does not promote honor killings and that the practice stems from sexism and tribal behavior that predates the religion" [2]. The idea of honor killings can take different forms, and it's not unique. Consider as well that many people are disowned by their families if they are gay. What is more though, the idea that this is a "non-western" morality is a extremely debatable, and is going after a small example to potentially emphasize confirmation bias.

- tradition over critical thinking: I listened to a preacher that came to my university college campus that lamented that there was no god in the university. That god should be taught in every class, physics & math included (which by definition those studies precludes god). The idea that the world can be left as an open-sewer, that the atmosphere cannot be ruined because God would not allow it and would rapture up everyone soon anyways - those are still pretty mainstream ideas in the West rooted in tradition. Consider yet more examples, "A League of their own" - traditionally women do not wear pants, yet any critical thinker would realize you want to wear pants when playing baseball. There's plenty of anecdotes and examples to put into question whether this is truly a western morality or not.

[1] https://www.pbs.org/wgbh/americanexperience/features/holocau...

[2] https://en.wikipedia.org/wiki/Honor_killing_in_the_United_St...


All of the above is to say, western morality is not all good (nor is it all bad), but the same goes for all the rest. To know more about further examples, that is an ignorance you need to work on personally to cure. Travel outside of the West & read more history. When travelling, culture shock is a real thing, it's striking to learn that others in the world actually think fundamentally differently, they have different values, and in some cases it's even a good thing (and sometimes not, but the fact it's different, it helps break the mold of your own experiences).


That's a very thorough reply and way more than I expected. I appreciate it, and it'll make me think more about this.


Thanks for sharing. I didn't realize how some people could justify things like factory farming and environmental depletion, and your explanation makes a lot of sense.


If that's the case, then OpenAI might very well be the new Netscape or Blackberry (RIM).


If capitalism driven climate change doesn't get us, capitalism driven AI is a good backup!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: