YourNetworkIsHaunted

Ironically I think it's also been discussed most frequently within Rationalist circles that these types of intelligence aren't often correlated. I'm not going to chase down links right now because doing an SSC archive exploration requires more mental fortitude than I currently possess, but I distinctly remember that a recurring theme was "if nerds are so smart why don't they rule the world?" In my less cynical days I had assumed that his confusion on this point was largely rhetorical, intended to illustrate some part of whatever point was buried in the beigeness. Now it seems like I was falling victim to the ability to project whatever tangentially-related thesis you want onto the essay and find supporting arguments because of how badly it's written.

Fuck it, I'm good to Gonch this out.

I forget, did we ever actually learn who the killer was in Murder at Wizard University? I remember it kept coming up through the first book as a kind of motif for how this new world wasn't necessarily as safe and clean as Tommy expected, but I think that whole business with the Thoughtknot ended up overshadowing it before the actual killer was revealed. Like, I get it thematically or whatever but it just stuck in my head as a loose thread and has bugged me for years.

On a purely rhetorical point, it seems like the whole counterargument from Gwern is just an argument-by-disorganization or something to that effect. He doesn't actually challenge the factual information presented, but does shift how those facts are framed and what the actual contention is in the background, and then avoids actually engaging with the new contention from the bottom up.

In a lot of discussions with singularity cultists (both pros and antis) they assume that a true superintelligence would render the whole universe deterministically predictable to a sufficient degree to allow it to basically do magic. This is how the specifics of "how and why does the AI kill all humans again?' tend to be elided, for example. This same kind of thinking is also at the heart of their obsession with "superpredictors" who can, it is assumed, use some kind of trick to beat this kind of mathematical limit in certainty (this is the part where I say something about survivorship bias). In the context of that discussion, the fact that a relatively simple arrangement of components following relatively simple, deterministic rules is still not meaningfully predictable past a dozen or so sequential events due to the magnification of the inevitable error in our understanding of the initial circumstances is a logical knockout.

Rather than engage with this, however, Gwern and his compatriots in the thread focus in on the tangent about how high-level pinball players are able to control for that uncertainty by avoiding the region of the board where those error-magnifying parts are. However this is not the same argument and begs the question of whether those high-chaos areas are always avoidable as they are in a pinball machine. Rather than engage with that question, Gwern doubles down on the pinball analogy, shifting the question even further from "how well can we predict the deterministic motion of a ball given the inevitable uncertainty of our initial state" to "how many ways can we convince a third party we've gotten a high score on a pinball machine". At this point we're not just moving the goalposts, we've moved the entire stadium into low earth orbit and gotten real cute about whether we're playing 🏈 or ⚽ football.

And given the conversation surrounding the thread and these topics on LW I'm not even going to assume that such a wild shift is the result of bad faith instead of simple disorganization and sloppiness of rhetoric. This is what happens to a community that conflates "it makes me feel smart" with "it actually communicates the point effectively".

A) At this point I would be more surprised to learn that AI psychosis wasn't infecting the upper tiers of the white house tbh. Like, at this point we could get a leak that Hegseth had been developing a literal god complex alongside his LLM mistress and I wouldn't bat an eye.

B) It seems like a particularly bad sign that this is coming from thr Saudis given that they've been a consistent ally that the US has spent a lot of material resources and political capital to support. Ed: not actually an official Saudi government source. When you assume you make an ass of yourself, etc.

I mean it looks kinda swastikesque imo, especially with the ambiguity over whether it's supposed to be one or two "I"s behind it. (In some cases it's FII with the second I split, and sometimes it's FIIInstitute with the top of the second and bottom of the third "I" visible).

I doubt they have the individual or institutional capacity to go after them in a timely and competent fashion, but there's plenty of time before August for someone to remind them about it, especially since this was a way for Anthropic and friends to reclaim some positive space in the news cycle. I can see some bad news for the bubble and/or war hitting in, say, June and causing Amodei to break out the "we stood up to trump" story again, which will in turn remind the dodderer-in-chief that they were gonna try and do something about that guy.

Part of what makes the RatFic version of this so weird imo is that despite being ostensibly rooted in relatively low-hanging fruit (e.g. what if we industrialized this pre modern setting, what if we rationally looked at the rules of this magic system, etc.) nobody other than the protagonist has ever thought about these things and even once the protagonist starts demonstrating some real world-conquering results (benevolently, of course) nobody ever really seems to want to copy their successes. Part of what made the actual industrial revolution unfold the way it did was because of the ensuing arms race of it. In addition to causing the lines on various economist's charts to go nearly vertical this also basically culminated in the first world war, which seems like the kind of event that they should be aware of. But of course in RatFic it seems like anyone who can't be talked around to joining up with our protagonist is too weak or woke or stupid to actually pose a threat to the Glorious March of Rational Progress.

You know, when I think about securely holding onto things and protecting them without damaging or dropping them, I think of a fucking OPEN CLAW said nobody ever.

I don't know, I think there's real value in allowing the public to keep an eye on aspiring supervillains.

I... "Fireproof steel I-beams" has to be taking the piss, right? Right???

Requiescat en urina, more like. Does that make the first of this generation of slopmakers to actually get shut down?

Oh sure, but when I send a cover letter where claude code told me about serious security issues and I used that knowledge to replace their internal app portal with my face I've "violated the computer fraud and abuse laws" or whatever.