I was a little shocked to learn about the undecidable Post Correspondence Problem. But when phrased about the unrecognizability of the complement, it is a lot less shocking.

There is generally always a mismatch between our intuitive sense of what an idea means and any abstract formal way to define an intuitive concept.

I would like to clearly explain the reason for this mismatch and find a way forward by working to create a framework where intuitive human-native concepts can be discussed with appropriate degrees of precision.

There are two or three kinds of (related but distinct) intelligence: speed, and correctness.

Correctness might be divided into accuracy (how good ones predictions are) and precision (how many things one is able to predict).

It is possible to have a very fast intelligence which cannot understand many things but is very correct about the things it does understand; it is possible to have a very slow intelligence which understands most things well enough to make decent predictions but has lots of error, or any other combination of traits.

In practice, if you are slow and correct, you are often thought to be dumb, and if you are wrong and fast you are also thought to be dumb. These are different failure modes, however.

• Consciousness has causal properties
• It is compatible with functionalist accounts
• It is fully physical
• It is not an illusion

In the 2000s, I was born, moved to the east coast, learned to talk then walk then run then read.
In the 2010s, I learned to ride a bike, I got a sister, I did eight years of grade school and half of high school, lived on a farm with goats, and came to love math.
In the 2020s (so far), I lived through a pandemic, met my future wife, lived outdoors traveling for five months including paddling from Canada to NYC, did undergraduate school, married my wife, had my first child, and started working on AI safety.

If all goes well I still get at least five more decades!

2030s
2040s
2050s
2060s
2070s

Things will stay interesting…

Most important book in my education might have been this one. Read it in 9th grade, was ready to teach myself calculus in 10th grade.

I’m like Cinderella
My wife is like the evil stepmother
And my son is like the stepsisters who get all the special treatment

A few predictions:

We are >10 years from being capable of building broadly superintelligent AI, even with real speedups from adding compute and leveraging capabilities of intermediate AI systems. About 75% confidence in the >10 years claim, inside view (before updating on what others think).

I do believe that if anyone builds it, everyone dies is basically true, but with a little bit of error around what everyone dies ends up looking like in practice.

However, I think warning signs will continue up to super intelligence, in other words even near-superintelligent systems will display alignment failures if we pay attention.

Conditional on this not happening, building superintelligence is no less dangerous; but this matters because it gives us more chances to stop increasing capabilities before it is too late.

Now depending on what the momentum is like stopping may be very difficult, as the exponential will probably be steep at this time; for this reason among others, it is always safest to stop now rather than later.

There is populism as in “charismatic public figure exploits the masses by feeding them ‘fuck the elite’ candy while robbing them blind” which is bad

And there is populism as in “let’s do what’s actually good for the masses, fuck the interests of the elite”

which is good, at least in medium-large doses