Speculative Ideas

Thought-o-meter

A pedometer/step-counter "is a device, usually portable and electronic or electromechanical, that counts each step a person takes by detecting the motion of the person's hands or hips".

A pedometer might be useful for people who actually care (and have a reason to actually care) about moving a sufficient amount every day.

On the other hand, it might be harmful if somebody gets addicted to it, allows their [not breaking of the streak] — itself a proxy for a sufficient amount of a good kind of mild physical activity — to take priority over other important stuff, like eating or sleep or time with other people or whatever. (See also: C. Thi Nguyen on Value Capture.)

So that's pedometer. That's one way to quantify some aspect of oneself to either gain insight into it just for the sake of getting insight or to use that new sensor to optimize that aspect of oneself. The benefits and perils are not unique to pedometers. I myself was at some point somewhat obsessed about counting my macros etc in Cronometer.¹

OK. Some people would like to optimize their cognition. You know, be more intellectually active, more creative, have more ideas in general, and a higher frequency of good ideas.

One might conjecture (i.e. I'm willing to conjecture) that a significant portion of the variance in the quality of people's intellectual outputs is about how much they are willing to think and the variance in that, in turn, is rooted in how much endogeneous positive reinforcement they are getting from thinking about some particular things. (To some extent it obviously works like: people are rewarding you for more of some kinds of thoughts and therefore you are going to be producing more thoughts like that and you're not getting much tired by continuing to produce thoughts like that.) This is partly from my diagnosis of my current bottlenecks: friction/high-activation-energy when I'm assessing actions like completing a mathematical proof, biting into some material to actually understand some subject, writing a post, getting around to edit an old document.

Would a thought-o-meter be helpful for that?

Sure, it would come with its own risks, including very close parallels of perils of pedometers: e.g. goodharting thinking to hit the minimum 1,000 "thoughts" per day instead of thinking for real, good, or fun.

But still, would a thought-o-meter be helpful for people who want to think more?

We obviously need to overcome some hurdles first:

My first thought was some kind of brain computer interface that would be able to read out the amount of novelty your mind produced based on the brainwaves or similar signal.

However, there's a simpler alternative idea: just have some file where you write down your ideas in a bullet point list and count the amount of new ideas you've head each day. Or you can have a CSV file with columns date (or datetime), name, description, context (e.g. in what situation you came up with this idea, what might have been the cause, whether you took some drug that you usually don't take etc).

The activity of thinking is not as straightforward as walking or doing some other kind of physical exercise.

The simplest way would be to just think in the direction you feel like thinking, like whatever's been recently on your mind: Proto-Indo-European mythology, your local Rasputin, or statistical modeling of juggling with Weibul distributions.

A qualitative hierarchy of kinds of uncertainty

There might be a way of organizing "qualitatively different" forms of uncertainty in a partial order or some similar sort of structure.

The lowest degree of uncertainty is lack of uncertainty, i.e. Boolean variables. Then, we have probability: a well-defined event space $W$ with a sigma algebra $\mathcal F\subset\mathcal P(W)$ over it and a probability measure $\mu:\mathcal F\to[0,1]$ such that $\mu(\emptyset)=0$, $\mu(W)=1$ and $\mu(\bigcup_iA_i)=\sum_i\mu(A_i)$ whenever $A_i$'s are pairwise disjoint. Perhaps we can modulate the (~quantitative?) degree of well-definedness/certainty by restricting $\mathcal F$ to be less than the whole $\mathcal P(W)$, in which case it can then be extended to all of $\mathcal P(W)$ with imprecise probabilities.

I'm not sure what to take as the next qualitative level of uncertainty. One option would be imprecise probabilities, i.e. "convex topologically closed $\bot$-closed subsets of subprobability distributions. Another option would be Dempster-Shafer belief functions (which are equivalent to lexicographic probability measures). If neither one of them is strictly stronger than the other in the relevant sense, we can fork our poset here.

Possibility measures and ranking functions seem to be hard to fit into this poset order. Maybe they should be independent branches rooted in Bool?

The top element would be some minimal plausibility measures and the element just slightly weaker than that would be relative likelihoods.

All of that still doesn't include:

• Logical uncertainty.
• Cases where the space over which the belief ranges is not defined (except perhaps something like all the things encodable as bistrings, a'la Solomonoff) or where it is subject to change.
• Anthropic uncertainty.

Salvage concepts

A salvage concept is a concept that is a remnant of a past way of thinking that has been overcome or that has failed to transfer/adapt to a novel intellectual environment but that the users of which have some stubborn attachment to it, motivating attempts to reformulate it in the new ontology in a way that is just sufficiently legit-seeming to seemingly justify (along with aforementioned stubborn attachment) letting it stick around.²

Examples:

• The conception of human soul as the essence of being has been (being) reskinned as sentience, capacity-to-feel/suffer, consciousness, whatever the mental/spiritual essence that people want to be a clean delineator between "things worth caring for intrinsically" and "things not worth caring for intrinsically".
• Moral realism and "one true morality in the world" have been reskinned as humanity's reflectively endorsed values or Coherent Extrapolated Volition.

(I can't think of other examples right now but I bet there's plenty more good examples.)

Related: zombie theories and shell games.

Wayback Machine Pastcasting

Pastcasting is forecasting the past, i.e. you are given some information about some past events until time $t$ and are supposed to, based on that information, to predict what would happen with respect to some specific thing at time $t$.

An obvious way to enhance this exercise/practice is to allow the user to access all the information on the internet published before $t$, perhaps also some sources about events before $t$ that are filtered for not mentioning any information causally downstream from what happened to the thing being predicted.

We already have a piece of internet infrastructure that can be used as a basis for this practice, i.e. Wayback Machine.

Recurring parallels pointing towards … non-foundationalism?

• Economics:
  • Orthodox neo-classical economics models economic actors as ~perfectly rational entities aiming to maximize their personal utility/gain, economy at equlibrium, etc.
    • They are often OK as approximations but they fail in situations of disequilibrium, etc.
  • Contrast them with [algorithmic/complexity] economics approaches like economy-as-ecology, simulations, bounded rationality, heuristics etc.
• Human cognition:
  • Human reason/[general intelligence] as a unified faculty versus a composite of specialized systems, as per The Enigma of Reason.
  • Human normative cognition as an elegant machine versus normative cognition as a kludge.
• Mathematics/logic:
  • David Hilbert wanted to found all mathematics on a set of axioms that was complete and self-provably consistent. Gödel showed that it was impossible. There's no way to "solve math".
  • (See also: Kaarel on infinite endeavors.)
• Artificial intelligence:
  • Systems ~hand-crafted from legible logic failed to scale, unlike organically grown matrices of inscrutabel floating-point numbers.
• Perverse Monisms:
  • Normative frameworks aiming to reduce all normativity/value to a single currency arise by capturing/collapsing/swamping whatever values humans start with.
  • More pluralistic systems like capabilitarianism allow for systematization of one's normative framework without risking that swamping.
• Software engineering:
  • Xanadu was aiming for a perfect coherence of the content. It failed. "Normal" internet is what it is but at least it didn't fail.
• Unified theories of how science universally works (when it works) versus understanding of science as a collective, organic process.
• One algorithm to rule them all vs metarationality.
• Live Theory.

Verbal expressions of intellectual (anti)complements

Different cultures/people embracing different intellectualist attitudes³ tend to differ in how (dis)praise somebody else for their cognition broadly speaking. Examples:

• "They are very smart/sharp." / "They must have a pretty high IQ/g."
• "They are a clear thinker." / "They are not the clearest thinker."
• "They are very emotionally intelligent." / "They have a high EQ."
• "They hold some fairly confused views." / "They are more confused than they should be."

Arguments from concept deficiency

There is a pattern of argument that goes something like:

I often found myself using this argument pattern but then realized that I am annoyed where people try to dismiss certain obviously relevant concept using this very same argument pattern or a very close one.

I think there is something to be salvaged about this but at the current moment I don't know how to fix it.

Qualitative and quantitative

From my LessWrong comment:

Plausibly one generally useful operationalization of the difference between qualitative and quantitative difference would be in terms of a function of some attribute of the phenomenon (or a collection of phenomena) that exhibits a phase transition when passing some threshold (possibly blurrily defined one, not necessarily a specific unique point), along with a specific reason why this function and threshold is the right operationalization in this specific context.

Inductive whac-a-mole arguments

In one of his uncountable podcast appearances, Connor Leahy, described "futurist whac-a-mole", a kind of inductive argument for why we should expect AI to cause a lot of problems.

To distill, abstract, and generalize, an inductive whac-a-mole argument is something like the following:

There is something satisfying about this argument. There's also something iffy about this argument in that it feels like it might be proving too much or be too general. We would like to strengthen it by introducing some additional assumption about the phenomenon at play, like that it is not practically exhaustive and patchable via ad hoc means but then the argument doesn't add anything, conditional on the assumptions of non-exhaustivity and non-ad-hoc-patchability. The argument might still be a fruitful argumentative or crux-identifying move: "If this has kept happening so far, why are you expecting it to stop happening at some point?".

Related:

• Sum-threshold attacks (TsviBT).

Overfitting the theory to the problem

There is one way of extending a theory (e.g. a mathematical framework) that I find particularly problematic. (Possibly "overfitting" is not quite the right word but I don't have a better one at the moment and I'm time-constrained.)

There's a problem $P$ that our current theory $T$ is failing to handle satisfyingly. So a theorizer is extending the theory $T$ to a new theory $T'$ such that $T'$ solves the problem $P$ in some more acceptable way.

However, the theorizer doesn't ask themself the question of whether the new entities or rules that were added to $T$ to obtain $T'$ are actually warranted or possibly real or meaningful or whether we have good reasons to expect them to be, in some relevant sense, admissible. The abstraction fails to be vetted.

Examples:

• Adding hyperreal numbers to handle some problems in population ethics.
• Tegmark IV (mathematical universe).
• Pancompuationalism. Whence the Turing machines that Run Reality according to Wolfram?
• String theories (?⁴).

Obviously, there are cases where a theory containing lots of theoretically posited entities that are at the moment of postulating the theory beyond the reach of the theorizer, is later validated.

This is not what I'm pointing at here. What I'm pointing at here is the theorizer having an unjustifiedly high confidence in their new theoretical extension.

One way not to fall prey to this is to check whether the constraints we're imposing on the theory pin it down uniquely (or perhaps at least some parametric-ish family of theories?).

Quoting from What's So Bad About Ad-Hoc Mathematical Definitions?:

A kind of wicked-problem-ness arises if the problem itself stems from some other confused theory $T^*$ in a way that makes it impossible to revise $T\mapsto T'$ or $T^*\mapsto(T^*)'$ individually but what is rather needed is an intelligent revision of a conceptual schema containing both $T$ and $T^*$.

Extensional reduction

Extensional reduction of a phenomenon X amounts to framing it in terms of its final/eventual/ultimate effects on the world, perhaps to how they are jointly determined by various properties of the phenomenon and the world.

It is akin to thinking of a computer program as a function mapping input $x$ to output $y$. Although, in a sense, this is what the program is doing, it is also doing more. There is a lot of hidden logic going on between $x$ and $y$. That logic is relevant to understanding how $x$ is mapped to $y$ and also to what happens when the clean functional abstraction of the program breaks down, e.g. when runtime errors arise, or if some hidden state of the system is changed and retained across executions of the program.

Similarly, the following extensional reductions miss out on a lot of the logic of the phenomenon.

A toy model of useful optimization

(Speaking very loosely here; a very toy model.)

Optimization operates within some constraints or structure specifying certain optimization channels. One way of thinking about this models constraints as fixed parameters and optimization channels as free/varying/adjustable parameters. Another way of thinking about it is: constraints as the set of hypotheses $\mathcal H$ and optimization as finding the best hypothesis $h^*\in\mathcal H$.

The purpose of optimization is to find something that conforms to certain fixed desiderata. The constraints don't "just" limit the optimization process, they also guide it so that it is more likely to produce the kind of result that the initiators of the optimization process would want.⁵

In general, for the pair (optimization-process,optimization-constraint) to produce a useful result, the two components need to be appropriately matched in their "power". If the optimization process is too powerful, it can find and exploit loopholes in the structure, finding the rare "technically legal/allowed" settings of the free parameters that score high on the "official" objective but are not desirable from the designer's perspective (see: specification gaming). The optimization process might even be so powerful so as to "escape" or "remake" the constraints (e.g. by exploiting some side channels⁶).

This might make it sound like the structure needs to be so powerful so as to always "resist" or "endure" the optimization pressure it will be subject to, but this is not always true. There are situations that call for what Dennett called for the problem solver to jump outside of the system (jootsing). For example, when the constraints need to be reconsidered or reimagined or when reification needs to be questioned, as is the case in decision theories that exploit acausal channels of influence (e.g. in newcomblike problems).

A "healthy" iteration of optimization processes involves a "dialectic" that keeps reshaping the pair of the optimization process and optimization constraints in a way that keeps delivering (possibly increasingly) useful results of optimization.

Examples:

• Various utilitarian frameworks might work well when you're considering a redistribution of goods/welfare within a specified population but break down when you start thinking about population ethics or intervening in the boundaries of the systems, so that you can maximize the stated optimization target by tiling the universe with hedonium (or similar).
• The paradox of choice is essentially about structure that is insufficient for the optimization process to find the right choice competently.
• Joe Carlsmith writes:

  • anti-realist rationality leaves the terrain it knows how to orient towards. Give anti-realist rationality a goal, and it will roar into life. Ask it what goals to pursue, and it gets confused. “Whatever goal would promote your goals to pursue?” No, no, that's not it at all.

• Civilization can be thought of as a set of constraints that allow for optimization to produce globally beneficial results.

External links

• Hyperphone (TsviBT).
• My idea of what the Christiano-Sisskind test might be.