• ref::Forcedness | MatthewBaggins

Forcedness

Meta: My current view is that forcedness is not the best lens through which to view these phenomena. More promising seem the two lenses that got developed in the course of this investigation, i.e. [the interaction between ambition and openness] and [the implicative structure of the space of minds in general].

What is forcedness?

Two attempts of gesturing at the maybe-phenomenon:

• Some contexts/situations force a mind to act/change/update in certain ways, even though it would intuitively/naively seem not to be necessary to act/change/update in those ways.
• Some decisions/choices/attributes of minds are forced. I.e. if the mind satisfies some set of criteria $C$, then it must also satisfy some other criterion/[set of criteria] $C'$.

Forcedness due to something else than boundedness?

Some cases of forcedness come down to generalized computational constraints. If, having an element A, I acquire another element B such that B does everything that A does and more, then I discard A because why would I unnecessarily clutter my mental space?

To the extent that forcedness is not just about that, i.e. sometimes one is forced to obey a criterion $C$ for reasons other than computational constraints, we need a different explanation of forcedness. One candidate for this explanation is that one property of the mind implies another property of the mind. If a mind has dynamics such that X, then its dynamics must also be such that Y. Having dynamics such that X and not-Y is either (a) impossible, or (b) self-destructive, prohibitively costly, vulnerable-to-exploits, or something like that; and to the extent that we expect strong minds to be self-destructive/not-vulnerable-to-exploits, they should obey that kind of implicative constraint.

Examples of forcedness

Some gestures/ostentions/tentative examples follow.

The Repugnant Conclusion

If you accept some families of normative premises plus the rules of inference, then you are forced to accept the normative conclusion that the world with 10^(10^100) barely happy lizards is better than a standard utopia.

Why are you forced to accept the repugnant conclusion? The mind is (i.e. you are) created already in motion. Implementation of dynamics inherent in the rules of inference comes down to accepting conclusions like this one.

We can try to work around by rejecting these rules of inference or some of the premises (see the discussion in the SEP entry). We can do that because the dynamics that (drawing) The Repugnant Conclusion relies on is not all of the locally relevant dynamics. At the very least, there is the implied repugnance (or aversion, or flinching away from the conclusion that seems/feels obviously/intuitively/viscerally wrong etc). The conflict between the (apparently valid) chain of reasoning and the repugnance that denies the conclusion induces questioning of the premises and rules of inference that [would not]/[may not] be questioned otherwise. (Similar to how deriving a falsehood from a conjunction leads one to question the conjuncts or even some parts of the logic itself (going paraconsistent?).)

Forcedness in this case comes down to (1) the impossibility of embracing the argument/conclusion along with its repugnance, and (consequently) (2) the need to "handle the conflict" by modifying one's "commitments".

One could always shrug and say "OK, but I don't care" and still prefer a utopia to the lizard world. This is tantamount to assuming that something must be wrong with the argument even if one can't exactly poitn out what is wrong with it.

Why is this relevant? Why can't we simply "separate our concerns" so that the abstract normative arguments live in one module and the more mundane ones in the other module?

An ambitious mind exploits all available influence channels to get more of what it wants. A repugnant-conclusion-like situation involves a tradeoff between the things a mind wants and/or indeterminacy of how much of [what the mind wants] would be obtained given such-and-such course of action. Closing this influence channel or patching it locally gets rid of the problem but also leaves potential indeterminate utility on the table.

A choice needs to be made. Incoherencies need to be ironed out. A mind may not know how to do it ex ante, in which case it needs to open itself to novelty in the form of something like new coherentifying procedures, natural selection-like phenomena etc.

Vagueness/The Sorites Paradox

(Structurally similar to the previous example.)

If you accept the two premises of the Sorites paradox plus the rules of inference, then you are forced to conclude that every man is bald. More generally, you seem to be constrained in the number of possible ways of resolving the problem(s) of vagueness, each of them being weird/bizarre/crazy/counterintuitive.

Cyclical preferences

If your preferences are $A\prec B\prec C\prec A$ such that you are willing to spend \1$to go$A→B$,$B→C$, or$C→A$, then a bookie can make you spend money running in circles as long as it wants.

Important background assumptions that may need explicitizing are:

1. Your preferences are defined over the set $\{A,B,C\}$, not trajectories between the elements of the set.
2. More money is better than less money: \n\prec$(n+1)$.

Therefore, if your preferences are cyclical, you are vulnerable to money-pumping. Since you prefer more money to less money, your preferences better not be cyclical.

On the other hand, if you already happen to be a mind with cyclical preferences and preferring more money to less money, you face the following, inherently open problem.

This is an inherently open problem because there is (probably) no "universally compelling"/"natural" way of computing that kind of tradeoff. In order to be able to assess the tradeoff "on your own", you need to have a way of doing that kind of assessment "built-in", even if implicitly (to be explicitized).

Otherwise, you need to "open yourself" to be modified by selection pressures (or something) that favor some ways of modifying you, e.g. because some "modified-you-s" are more adaptive than others.

Mind uploading

Humans have very robust intuitions regarding what entities in the past, present, and future they identify/recognize themselves as. At any moment in time, there is either zero or one person who is me and this person shares with [present me] a cluster of very highly correlated features, most notably various kinds of continuity (physical, psychological, social, behavioral).

Now, what if I upload my mind? Is my upload "me"? How is the answer modulated by such as variables as whether the original me survives, whether me and the upload diverge in our values/characters/[social roles], whether there are more uploads, and so on? Do I need to make a choice as to structure/define my caring about those minds? How principled does it have to be?

Why may this question be important? When thinking forward/planning, one thinks about [future oneself] differently than about [future other]. This difference consists in (at least) having greater control and predictive power over [future oneself] than over [future other], as well as caring more about [future oneself] than about [future other] (for most [future other]s).

One might privilege one's original/physical/biological self but what if there is no such self or there are many such selves/clones indistinguishable from each other? One cares more about oneself (or one's past/future selves) than about random strangers. Most well-functioning minds most likely do so too (at least isntrumentally). To act coherently in such a (strange-seeming to us) setup, the notion of selfhood needs to be extrapolated. Intuitively it seems that transitioning from a simple setting (one self through time) to this new bizarre setting while preserving one's prior values/goals necessitates appropriately extrapolating the notion of selfhood. It is neither clear what is to be denoted by the labels "appropriate extrapolation" or "value/goal preservation". What if a value/goal is tied to something contingent to the current one-self-through-time setting and becomes irrelevant upon transitioning to the new bizarre setting? (Cf. concept extrapolation, ontological crises.)

Previously I said:

Expanding the domain of discourse to uploads etc makes it not binary/less binary (there may be more than one me at time $t$) and potentially more fuzzy (if I have one upload at time $t$ and this upload diverged from "me" such that it makes sense to say it's "50% me"). It could be fuzzified even without introducing such kinds of paradoxes, e.g. ground the notion of personal identity in a combination of memories, values, and social roles, so that we can say "If 'my 90-year-old self' will have 80% of memories, values, and social roles of the current me, then that person will be 80% me.". That fuzziness may be indispensable when we are forced to deal with uploads.

Moral patienthood

A "moral patient" is an entity that features in our moral considerations, i.e. something or somebody whose well-being/interests/preferences/rights etc we take into account. (Sometimes the term "moral status" is used, see SEP.)¹ By default, we do not have a strict definition of moral patienthood. We have some general, not [consistent/unified/systematic by default] rules for determining it, e.g. some attributes/properties of an entity incentivize us to attribute moral status to that entity. Some people are "into systematizing stuff" (though this description probably is not complete) and therefore inclined to try to [explicitize what should guide them in determining moral patienthood] and [play with various ways of extrapolating/systematizing their concept of moral patienthood].

In any case, suppose we have some (perhaps complex and convoluted) criterion by which we assess the (plausibly multidimensional) moral patienthood of other beings. The criterion is well-behaved on some set of "inputs"/"cases", i.e. it gives clear answers on that set. Suppose the criterion is given some weird entity X and it "doesn't know what to output". What should it do?

Here again being ambitious (in this case manifested as caring about moral patients) forces a mind to interact with novelty in novel ways. This is an "external reason" to be less ambitious, i.e. it's a maybe-adaptive-pressure on natural selection or whatever other mind-generating process to produce a mind that is not that ambitious (at least not that ambitious in this particular way). This is opposed to an "internal reason" by which a mind volitionally guides its actions.²

The St. Petersburg Paradox and other decision-theoretic exploits

Suppose you follow some decision theory (DT) in its totality, i.e. use it as the algorithm for determining your actions, given your best estimate of some state of the world, your preferences etc. There is no "outer loop" around it, no external reflective algorithm wrapper that monitors the internal DT and steps in when things seem to be approaching a precipice. If your DT has some exploits (e.g., The St. Petersburg Paradox) then, if you encounter an exploit, you are guaranteed to fall for it. So you better implement a DT that doesn't have any exploits that you are likely to encounter, given your best guess of what kind of situations you are likely to find yourself in (although this is now becoming circular). (Cf. Eric Schwitzgebel on formal decision theories being optional.)

Bistable figures

Take Necker's cube as a typical example. You see a 2D image that can be interpreted as a representation of a 3D structure. It's ambiguous. There are 2 possible interpretations of the image as a 3D structure and you can only "see one 3D thing in the 2D thing".

Human³ visual system learns to associate certain 2D projections on the retina with certain objects in the 3D space. The mapping 3D→2D is not one-to-one, so sometimes we need to be able to associate one 2D projection with at least two 3D objects. This is particularly pronounced when the 2D projection is the only information available as in the case of Necker's cube. When there is no additional information, mere idle-musing-ish cognitive fluctuations can be taken ([are taken]/serve) as additional information. That's why which face of the cube appears/[is being interpreted] as the [front/back] one seems to be flipping back-and-forth and controlling it is to some extent learnable.

Something similar is the case for other multistable percepts, e.g. binocular rivalry.

Gestalt-switching is a plausible generalization to more high-level cognition, with evolutionary transitions being one specific example: an evolutionary transition occurs at many levels but you can't keep all of them in mind.

Theory change

Suppose I accept Newtonian Mechanics (NM) as a theory of physics, using it to make predictions about physical systems whenever I have an opportunity to do so. At some point I learn about General Relativity (GR) and realize that NM actually is not the fundamental theory, as it can be derived from GR, a more general, powerful, and accurate theory of physics.

What changes about my epistemic state? It seems like the dynamics of my mind are such that I want to cut to the core abstraction describing my reality and so I am compelled to kind-of-relinquish NM as the theory that I see as "basic" in the sense of [grounded in evidence-delivering practice, ungrounded in theory].

What if I don't want to relinquish NM "in favor of GR"? Maybe I attach something like terminal value to "considering NM for all practical purposes the fundamental theory of physics at least in its domain of application"? Whenever I can predict a system using NM, I use NM. If I can't (or I'm in doubt whether I can), I use GR. NM plays the same role in its local context as it has played before I learned about GR. The NM-neighborhood part of my current mind is Gemini-modeling the corresponding part of my pre-GR mind. I value having this part of my mind do this.

But how is it any different from the alternative situation, where NM "cedes the ground" to GR? In practical situations, I use NM anyway as a good-enough approximation. In order for this distinction to matter, I would need to attach value to minor patterns in the structure of my mind, basically differences in "mere labeling" (FWIW), rather than connections. Maybe it is possible/tenable for a mind to be shaped like that but it seems to me to be very implausible and I don't expected most minds to have that shape of caring.

Is this kind of theory change an example of forcedness? What exactly am I forced to do? Being a kind of mind that tries to understand physics, I notice (I'm forced to notice?) and incorporate the relationships between NM and GR. I continue to use NM as a heuristic/approximation anyway. I may attach value to using it but it doesn't change anything. To the extent that values are about Things, this value of mine is not about a Thing. (Are values forced to be about Things?)

Viewed like this, NM-GR does not seem to me to be an example of forcedness; or if it is, it features forcedness in ways different than the previous examples.

Change of faith

(Probably analogous to the previous example.)

My friend J told me about their departure from Christianity. It was a slow drift-off that eventually culiminated in realizing that they "live exactly as an atheist with a given morality would live" and they merely kept considering themselves as a Christian. The realization of [generalized behavioral] equivalence of their behavior with the [atheist version of J]'s led them to drop the thing.

Isn't this exactly analogous to the NM-GR case? If not, what are the differences?

I see two ways forcedness may manifest here and both are analogous to the NM-GR case. First, the mind changes its conception of what is good/[should be done], somewhat (suffifciently) similar to updating its theory of physics. Second, the mind cares about the structure of its caring, not about the labels of caring, and once it realizes that the labels relating to caring are failing to track the structure of its caring, the labels are rearranged to track the structure better.

Some alternative speculations (written before I came to lean the above):

An attempt to distill the essence of forcedness

What unifies these examples? Can we generalize?

Is "distilling the essence" the right metaphor/[way of thinking]? Does it encourage some too-static-and-context-independent ways of thinking? (h/t Sahil)

I currently see two lenses through which this could be viewed. It would be nice if they were complementary, although right now I don't quite see whether they complement each other.

Implicative structure of the space of minds-in-general

The mind is created already in motion. To be any mind is to implement mind-dynamics. To be a specific type of mind $X$ is to implement a more specific kind of mind-dynamics characteristic of $X$ and implementing $X$-dynamics has certain implications for how the mind interacts with novelty.

We can distinguish between:

• Generic forcedness — what any mind is forced to do in virtue of being a mind at all.
• Specific forcedness — what a mind of some specific type is forced to do in virtue of being a mind of that specific type.

A salient aspect of forcedness is that we are not primarily interested in forcedness due to computational constraints, boundedness, scarce resources, etc. But [what type of mind one is] is co-constituted by one's embeddedness. Therefore, constraints etc are co-defining [what type of mind one is].

Are ambitious elements an important piece of mind-dynamics?

One "mechanism" behind this kind of forcedness is ambitious elements. An element's (e.g. a concept's) ambitiousness may manifest in its trying to apply itself to the entirety of its available domain. If you had an ontology that carves the normal stuff into one domain and the weird alien, mind upload-y stuff into another domain (and assuming that the two elements accept that ontology) and the two elements see themselves as assigned to those two different domains, then they would not be in conflict.

But such a "separating" ontology would not be stable if an insight was gained into the deep connection between the two realms. Something along the lines of "a mind created already in motion will likely integrate this insight into its prior ontology".

To the extent that it makes sense see such elements as somewhat agentic, maintaining a stable equilibrium between them would require stable maintanance of partial agency.

This still doesn't explain why we should expect elements such as the concept of personal identity to be ambitious? We might conjecture (at least in this specific scenario) something like the following.

The concept of personal identity is value-laden (moreover, deictically value-laden; not clear whether that's important here though). A value-laden element, combined with understanding of its domain of applicability, has obvious reasons to want to apply itself to the rest of its domain. The system-as-a-whole may try to contain/control/subjugate this ambition, (similar to how the Darwinism of individual cells has been mostly successfully subjugated by multicellularity; or maybe cosmopolitan-Leviathan more broadly).

Is there a reason to expect this kind of subjugation to be very difficult/unlikely/unnatural?

Elements such as "the concept of personal identity" may be totalizing in that they are relevant for a vast majority of what the mind might care about. This is not the case with individual cells or even most humans. They do not have enough intelligence/[optimization power]/oomph to initiate a wildfire of strategicness. Importantly, the concept of personal identity is not all that's required for the mind to coherently actualize its values. Thus, it will form coalition with a selection of other "specialized" elements, each having different, complementary roles. (It's possible that the decomposition of the mind's pursuit into those roles has some degree of arbitrariness/history dependence.)

Ambition encountering radical novelty necessitates openness

An ambitious mind exploits all available influence channels to get more of what it wants. Sometimes it is not clear how to trade off between various things being wanted. Sometimes it is not clear whether an X is actually what a mind and this cannot be resolved by gathering more information (or incorporating novelty in more usual/default ways). Sometimes whether X is better than Y is indeterminate, again in a way not resolvable by gathering more information. Call that kind of situation "radical novelty". (These are likely other kinds of things that should fall under that umbrella.)

When an ambitious mind encounters radical novelty, it is presented with a handle, i.e. a potential channel of influence. The problem is that the value of this handle is indeterminate. It has a fuzzy choice between leaving "utility" on the table and making use of the handle, even though it is not clear how to assess its value.

Some questionable and very uncertain propositions follow:

• The more ambitious the mind is, the more it will lean to using the radical novelty handle.
• To use the radical novelty handle, the mind needs to incorporate radical novelty. Since this novelty is radical, it can't incoroporate it "on its own", i.e. using its "internal machinery" or anything it can acquire by incorporating/[interacting with] non-radical novelty.
  • Therefore, to use the radical novelty handle, a mind must open itself to be changed by radical novelty.
    • To the extent that being ambitious necessitates being (a part of a lineage that has been) changed by opening oneself to radical novelty, steam/[determination bias] may be a relevant factor.
  • To the extent that this breaks down the mind's coherence/alignment with its pre-radical-novelty self and this is detrimental to something like time-and-identity-extended goal-pursuit, this interaction between ambition and radical novelty may put a pressure on the mind's telotect not to make an overly ambitious mind.

Footnotes