Ambition, Openness, and Radical Novelty

Central conjecture: Maintaining high ambition in the face of radical novelty necessitates opening the target of ambition.

Provisional definitions of the terms:

• Ambition — Willingness (/tendency/wanting) to touch a lot of the Cosmos to satisfy one's purpose.
  • Target of ambition — The goals that ambition is about.
  • The most central distinction between low- and high-ambition can be seen in the context of multiply realizable goals. Suppose the only thing you want is diamonds. If you are low-ambition, you are satisfied with one diamond, or maybe a ton of diamonds. If you are high-ambition, you will strive to rearrange your entire lightcone into a diamond factory.
• Openness — Being available to making novel relations,
• Radical novelty — Novelty that fundamentally puzzles the mind. Novelty that makes the current elements used by the mind inapplicable even though they should (in some appropriate sense of "should") be applicable.
  • This is especially relevant when the elements in question are value-related/value-laden.
  • Mundane — The domain one is acquainted with and whose elements (e.g. values) are well-defined over.
  • Arcane — The domain one is not acquainted with and whose elements (e.g. values) are not well-defined over.
• Values — The wanting-ish things that specify (?) what a given mind wants to achieve/[make happen in the world]. (Very provisional concept.)

General picture

(Very abstract, sparse in examples. More examples will likely follow in some unspecific future.)

A mind faces radical novelty, when it encounters a situation such that its prior elements (the structure incorporated so far) cease to apply. The mind can walk away or keep facing radical novelty (or some combination/in-between). The more ambitious the mind is, the more it is inclined to attempt to exploit the radically novel influence channels, even if it means the need to face radical novelty.

In order to exploit a radically novel influence channel, its novelty needs to be incorporated. Since that novelty is radical, the mind needs to be restructured so as to be able to handle that novelty.

What is that restructuring? By supposition, the mind's values are well-defined over the mundane (proportional to the mind's ambition). The values are not well-defined over the mundane. There are many possible paths of value/concept extrapolation, leading to different conclusions/implications about what is the right thing to do. The more ambitious the mind is, the more will it strive towards getting more of what it wants. However, when it is not clear what it wants, as is the case when facing the arcane, it can't get what it wants.

There is no universally privileged null action. Deciding that the arcane doesn't matter and values don't terminally apply to it is a choice.

There is no universally privileged [naive default]/[maximally ignorant] action. Doing some kind of averaging across possible extrapolations is a choice and there are good reasons to expect that it is not a good choice in most cases.¹

So an ambitious mind needs to "make up its mind". It has to "decide" how to extrapolate/refine/modify its values such that they well-apply to the arcane.

How can a mind do that? If the mind has some satisfying idealization/reflection procedure, that procedure can be applied. It is an open question whether in worlds similar to ours in complexity/richness, a procedure can exist that deals with all the novelty a mind may encounter (I am doubtful).²

Absent such a procedure, what can a mind do? If it cannot decide between possible extrapolations, it undergoes something similar to decisional agonizing. But if its ambition is strong enough, it needs to decide, even when it can't.

Hypothesis: If its ambition-to-do "overcomes" its inability-to-do, it will open itself to "external" influences that modify its values so as to make them well-defined-ly achievable in the arcane (or at least given that the arcane exists).

If this analysis is onto something, that something may be a core mechanism of what I've been for some time calling/[thinking of as] ​[externalized shouldness]/[deference to Telos]. To the extent that humans, throughout their mental development, encounter small pockets of arcaneness, they need to re-open themselves/[their values] to external influences, at least unless they have something like the aforementioned "idealization/reflection procedure". Even then however, that procedure may be viewed as a consistent external influence. It is not meaningfully different from a surgeon that is always with you and fixes your brain when your ambition comes into tension with your lack of normative clarity.

How should we think of what happens to the mind when it opens itself up in the way just decribed? One possibility is to frame it in terms of "the forces of selection", a competition between different possible extrapolations (prospective elements?). In full generality, the selection framing doesn't seem quite right. Typically (at least as far as I can introspect), opening up one's values when faced with the arcane looks more like trying to navigate some abstract space of reason(s) that will seemingly dictate/determine the right path of extrapolation. (The phenomenology (?) of this is an important avenue for further investigation.)

In a sense, the space of reason(s) is kind of hallucinated/imaginary. Unless you assume some kind of mathematical platonism, "it doesn't really exist". It's more of a mental structure composed of abstracted summaries and inference tickets that the mind has learned. Navigating this structure is a kind of search for the correct thing, whatever that correct thing actually is. When there is no such correct thing but the demand for it is quite high (i.e. high ambition), the structure needs to be expanded or modified in order to make such a right thing appear.

In the process of adapting the values to the arcane, the way the values apply to the mundane may change.³ Whether this occurs probably largely depends on the degree to which the mind expects its values to [latch onto]/[refer to] some Things that transcend the mundane/arcane boundary.

On the other hand, if a mind is low on ambition, then it moves away from radical novelty, maybe even actively avoids it, sticking up to a subspace where its elements are well-applicable/at-home.

Questions:

• How often/when do minds in our world face radical novelty?
• How often/when does that kind of restructuring touch the mind's values?

Examples:

• A morally ambitious mind faced with radical novelty in the problems of moral dilemmas (dilemmas according to its current normative views), refactors its current values. To refactor those values it opens itself up to the hallucinated space of reasons in the form of abstract somewhat-logical arguments. If the mind is not so morally ambitious (though its ambition may be high somewhere else or it may be morally ambitious in some other way, emphasizing other radical-novelty-dilemmas), then it shrugs and prefers not to [think about]/[deal with] those problems.
• (More will follow.)

Steam/determination bias

A factor that may be relevant is steam/determination bias.⁴ Assuming tha this abstraction is onto something important, high ambition of some goal-pursuit seems to imply high steam in that goal-pursuit and consequently the belief/perception/evidence that the mind has a history/[track record] of pursuing that goal effectively.

To the extent that the evidence [contained in]/[referenced by] the mind's current steam is up-for-interpretation, the mind may (likely will?) interpret it in ways that are more "favorable" for the purpose of dealing with the current radical novelty. If the mind has a history of ambitiously facing radical novelty in this way, this may incentivize the maintenance of some slack buffer (soft optimization?). This would mean that in order to go high-steam globally, we need to consistently go low-steam locally.

Low-ambition pressure on the telotect

In a world abundant in radical novelty, high ambition may be maladaptive, i.e. it may consistently lead minds to develop in ways that make them act cross-purposes throughout developmental time. To the extent that the [mind's]/[minds'] telotect can "learn" from such feedback, it will update toward producing minds that are less ambitious or [ambitious in ways that don't reliably lead to maladaptive development].

Implicative structure

All of this seems to be a specific case of the implicative structure of the space of minds-in-general. I.e. if a mind is of type $X$, then it is also of type $Y$, where "being of type $X$ or $Y$" ought to be understood very liberally (i.e. it may include features contingent on the world, relationships with other things in the world, etc).

The specific implicative structure discussed here is framed in terms of the relationship between ambition, openness, and radical novelty; and secondarily also some other things.

Further directions

• It feels like there should be some interesting relationship between this and partial agency.
  • Prevalence of radical novelty → tradeoff between ambition and [high goal-content-integrity]?
• What exactly is this "hallucinated/constructed space of reason(s)" that an ambitious mind needs to open itself to?