rationalism AI cultist creeps

Looks like the myriad achievements of poor HAL are ignored as he is shoe-horned into being the latest of a long line of ILX strawmen.

http://singularityhub.com/about/
http://lukeprog.com/SaveTheWorld.html

Hardware and software are improving, there are no signs that we will stop this, and human biology and biases indicate that we are far below the upper limit on intelligence. Economic arguments indicate that most AIs would act to become more intelligent. Therefore, intelligence explosion is very likely. The apparent diversity and irreducibility of information about "what is good" suggests that value is complex and fragile; therefore, an AI is unlikely to have any significant overlap with human values if that is not engineered in at significant cost. Therefore, a bad AI explosion is our default future.

its deeply weird to me how much of this stuff is out there, and how much is fixated on the idea that superintelligent machines are coming soon and the big problem is making sure they don't decide to kill all humans.

* How can we identify, understand, and reduce cognitive biases?
* How can institutional innovations such as prediction markets improve information aggregation and probabilistic forecasting?
* How should an ethically-motivated agent act under conditions of profound moral uncertainty?
* How can we correct for observation selection effects in anthropic reasoning?

http://www.fhi.ox.ac.uk/research/rationality_and_wisdom

how much is fixated on the idea that superintelligent machines are coming soon and the big problem is making sure they don't decide to kill all humans.

Some of us were traumatised by Servotron at a young age, OK?

"how much of this stuff is out there" : the big ideas are made by the same few people (yudkowsky, maybe bostrom) and the evangelization is made by about a dozens younger "lesser names" (that probably were hanging out on the sl4 mailing list) on 3 or 4 of their platforms that they rename / shuffle around every few years. they were "theorizing" about friendly a.i. way back then, i doubt they made any breakthroughs since then... how could they?

in a way the "friendly a.i." advocates are like the epicureans who 2300 years ago conceptualized the atom only by using their bare eyes and their intuition: some time down the line we sort of prove them right but back then they really had no good understanding of how it worked. who knows in the (far) future it's possible some stuff they talk about in their conceptualizaiton of a friendly a.i. will be seen as useful and recuperated .

Some people familiar with the LessWrong memeplex have suffered serious psychological distress after contemplating basilisk-like ideas — even when they're fairly sure intellectually that it's a silly problem.[5] The notion is taken sufficiently seriously by some LessWrong posters that they try to work out how to erase evidence of themselves so a future AI can't reconstruct a copy of them to torture.[6]

http://rationalwiki.org/wiki/Roko%27s_basilisk

In LessWrong's Timeless Decision Theory (TDT),[3] this is taken to be equivalent to punishment of your own actual self, not just someone else very like you — and furthermore, you might be the simulation.

sounds like a great theory to have, sooper sound

if you can believe that life is a simulation where you may be punished you might as well believe that there's a god who wants you to act a certain way

and i thought these people were only slightly crazy

at least they seem to be nominally self-policing

i mean tdt and rokos basilisk basically turns the AI into an arbitrary and all powerful god, if it wasnt that already

A god who wants you to send all your money to a particular organisation - so Yudkowsky is the televangelist in this scenario.

Yeah it is interesting that a bunch of self proclaimed rationalists have managed to reason themselves into a belief system functionally identical to one they would decry as irrational.

i love it! its borgesian

yeah i think the aporia there might be the "self proclaimed" bit

If I were a future AI, I'd recreate the LessWrong memeplex, except I'd call it LessRong. Why? For the lulz...

Roko's basilisk sounds like the name of an Italian prog rock band.

Roko's basilisk is notable for being completely banned from discussion on LessWrong, where any mention of it is deleted.[4] Eliezer Yudkowsky, founder of LessWrong, considers the basilisk would not work, but will not explain why because he does not consider open discussion of the notion of acausal trade with possible superintelligences to be provably safe.

that wiki article is like half of a great ted chiang short story

Found the following beautiful sentence at the bottom of the LessWrong page:

The basilisk kerfuffle has also alienated fellow cryonicists.

Why a basilisk?

http://en.wikipedia.org/wiki/David_Langford#Basilisks

I'm not sure you should give these guys what they want and proclaim them to be the vangaurd of Hard AI proponents... I really don't think anyone who has seriously grappled with the philosophical implications of, say, the physical symbol system hypothesis, could ever proclaim any development or avenue of research to be "provably friendly."

Furthermore I think its not very fair to suggest any and all fans, theorists or proponents of AI are similarly robotic in their thinking as these LessWrong people.

this is the best part

[T]here is the ominous possibility that if a positive singularity does occur, the resultant singleton may have precommitted to punish all potential donors who knew about existential risks but who didn't give 100% of their disposable incomes to x-risk motivation. ... So a post-singularity world may be a world of fun and plenty for the people who are currently ignoring the problem, whilst being a living hell for a significant fraction of current existential risk reducers (say, the least generous half).

because it means that one of the things that caused "severe psychological distress" was the suggestion that posters on rationalism message boards would in the future be punished for being smarter than everyone

what a terrifying perversion of one's value system

These fools are the enemy of the true cybernetic revolution.

ok sam harris doesn't really belong here but c'mon

http://www.samharris.org/blog/item/free-will-and-the-reality-of-love

Consider the present moment from the point of view of my conscious mind: I have decided to write this blog post, and I am now writing it. I almost didn’t write it, however. In fact, I went back and forth about it: I feel that I’ve said more or less everything I have to say on the topic of free will and now worry about repeating myself. I started the post, and then set it aside. But after several more emails came in, I realized that I might be able to clarify a few points. Did I choose to be affected in this way? No. Some readers were urging me to comment on depressing developments in “the Arab Spring.” Others wanted me to write about the practice of meditation. At first I ignored all these voices and went back to working on my next book. Eventually, however, I returned to this blog post. Was that a choice? Well, in a conventional sense, yes. But my experience of making the choice did not include an awareness of its actual causes. Subjectively speaking, it is an absolute mystery to me why I am writing this.

this is sub david brooks

this is not going to shock anyone but this frame of mind/crew of people trends very strongly into some supremely nasty politics

http://lesswrong.com/lw/hcy/link_more_right_launched/

ahahaa "Just so long as we don't end up with an asymmetrical effect, where the PUAs leave but the feminists stay."

ah god i don't think i've seen the term "race realism" before

The all-important gap between labeling yourself as a rationalist and actually using your reason; between labeling yourself as an empiricist and actually studying phenomena.

first thing i do after the singularity: allow myself to get a girlfriend! (i have actually read that from one of the big cahuna in a chat years ago. screencapped it but decided not to save for luls, i'm not that kind of guy)

next sentence written in the chat was of him again : "that was dumb."

everyone will be your girlfriend after the singularity iirc

also that basilisk thing is o_O

not least because it apparently relies in part on some absurd population ethics ("total utilitarianism" they call it)

http://lesswrong.com/lw/hcy/link_more_right_launched/

― R'LIAH (goole), Thursday, August 1, 2013 7:13 PM (Yesterday) Bookmark Flag Post Permalink

Why did you get me down the rabbit hole of a right-wing blog?

If there are multiple cultural/ethnic identities, they need to be either assimilated into one another, be distinctive and have clear guidelines for interaction, or be separated and with separate administrative structures.

i jumped ship at "race realism"

i think they are confusing cultural/ethnic identities with member planets of the federation?

also from that blog

When it comes to art and music, thinkers intuitively realize that the most popular works are the most trivial and idiotic, but when it comes to politics, the uninformed opinions of the masses are placed on a pedestal. The reason for this inconsistent view of a sort of Democratic pseudo-religion that has been in place in the Anglosphere since around 1848.

Advocates of Democracy try to rewrite history and imply that Enlightenment principles are fundamentally incompatible with Monarchy, but this is clearly untrue. Voltaire, known as one of the greatest thinkers of the Enlightenment, had a close relationship to a number of monarchs, including Frederick the Great, and advised him regularly. It was economic and cultural flourishing brought on by absolute monarchy in France that created the conditions for the Enlightenment and the Scientific Revolution. All of this was underway well before the French Revolution.

what monarchy do you want!? do you want to join the commonwealth!?

For roughly 165 years (since 1848), democracy has caused social and economic mayhem worldwide. Rule-of-the-People has caused vastly increased crime (100X in the UK since 1800)

ladies and gentlemen, statistics!

i think it's pretty obv that these people are far less smart and rational than they claim to be and after that by their own rules we're okay to ignore them

aaaand now we're moving into infowars territory....

Speaking for myself personally, my key motivation is not having to witness or experience global nanowar. For a grasp of the capabilities that could be invoked during such a war, I recommend the obscure volume Military Nanotechnology: Potential Applications and Preventive Arms Control.

It’s laborious for me to explain why small robots would be a major risk, because it should be self-evident. Very small robots could be made exceedingly stealthy, they could provide comprehensive surveillance of enemy activities, and could inject lethal payloads of just a few microliters. Moreover, they could self-detonate after carrying out their mission, making them untraceable.

uuuughhhhh ok if i HAVE TO EXPLAIN IT TO YOU

lol goole you have the weirdest hobby

It was economic and cultural flourishing brought on by absolute monarchy in France that created the conditions for the Enlightenment and the Scientific Revolution. All of this was underway well before the French Revolution.

Yeah but the computer this guy is typing this on is a product of capitalism proper which is only possible after the French Revolution and the death blow it dealt to the vestigial feudalism

Also questionable whether absolute monarchy 'brought on economic flourishing'? Weren't the poor in the period leading up to the Revolution having to subsist on grass and hay?

I'd love to be clever enough to write an algorithm that measured the ratio of pro-reason rhetoric to actual chains of reasoning in all forum posts and comment boxes on the internet

Which is just a fancier way of saying "the end justifies the means". But your chosen formulation of it immediately suggested the thought that consequences are open-ended, extending into all futurity, and therefore are impossible to measure

I think "the end justifies the means" is a bit more slippery - it's often used to weigh one set of consequences more heavily than another, e.g. bombing hiroshima to end the war. And, well we're talking about human actions and human consequences, I think its fair to restrcit it to humanly measurable ones.

Even human consequences extend indefinitely. Identifying an end point is an arbitrary imposition upon a ceaseless flow, the rough equivalent of ending a story with "and they all lived happily ever after".

so do you never consider the consequences of your actions or do you have trouble getting up in the morning?

I am not engaged in a program of identifying a universal moral framework based upon the consequences of my actions when I get up in the morning, which certainly makes it easier to choose what to wear.

touche!

This is the ideal utilitarian form. You may not like it, but this is what peak performance looks like pic.twitter.com/uHvCp2Cq7y

— MHR (@SpacedOutMatt) September 16, 2022

incredible

Read this a few days ago. As AI burns through staggering amounts of money with no reasonable use case so far, all your fave fascist tech moguls are gonna hitch themselves to a government gravy train under a Trump administration (gift link): https://wapo.st/3wllikQ