V-*


Meta

  • This is a somewhat disorganized brain dump of me trying to decompose value-related concepts. I consider myself to have failed to do this in a satisfying way but have failed in an informative way.
  • I'm putting load-bearing concepts/terms/expressions that resurfaced in the analysis into "these brackets".
  • This is still work in progress and will likely be edited a few times, so be aware of this if you feel inclined to hyperlink to it.
    • In particular, it treats V-stuff as "merely useful predictive/modeling" devices. This is pretty weak. I want something stronger. Perhaps I will find a way to come up with this "something stronger" once I analyze the concepts that came up in this analysis.

Intro

Daily language1 contains many terms/concepts/expressions pointing at the normative aspect of a mind/agent. I'm going to refer to them as the "V-cluster" (or "V-stuff", "V-things", etc). Here are some subclusters I obtained by grouping them based on some features that seem salient to me:

  • Referring (trying to refer?) to some internal properties of the system that we use to explain its otherwise perhaps mysterious behavior (and the ones I think most about)
    • "preference", "goal", "purpose", "objective", "value", "care", "like"
  • Normative/action-related directedness with a possibility of flaw
    • "try", "aim", "pursue", (perhaps less centrally) "intention"
  • Something internal to the agent but (connotationally?) more "detached" from it
    • "desire", "motive", "drive"

Aside from that, we have concepts of "utility function", "terminal goal", "instrumental goal", and "preference" is the more specific neoclassical-econ/axiomatic-rationality that is (perhaps?) a somewhat skeletal operationalization/[attempted explication] of the daily language notion of "preference".

In this analysis essay, I'm going to analyze these concepts with the purpose of understanding their connotations, denotations, inter-relationships, and dynamics of their participation, as they feature in my conceptual web (which plausibly is not that much of an outlier relative to this writeup's readers).

First, however, I would like to analyze the V-cluster in general. What are the roles that the V-concepts play in our language? How do they interrelate with other concepts and assumptions in our network? What dynamics do they induce in our reasoning?

However, in the process of trying to do so, I actually realized that it makes sense to start with the notion of "preference" as a particularly relatively crisp example, then generalize from it to V-in-general and then specialize back to particular V-terms. I do this because it turned out that as I intend to dig deeper into the notion of "preference", I keep uncovering its less central uses/aspects and thus bringing to the foreground the blurriness of the distinction between preference and other V-terms.

Preference

In economics and axiomatic rationality, preferences are an ordering over some… stuff, that we call "the domain of preference". These may be, depending on your framework of choice, propositions, actions, outcomes/events, etc. This ordering is typically taken to satisfy some properties, such as transitivity (which makes it at least a partial order) and completeness (which makes it a total order). The domain of preference may include/[be extendable to] lotteries over these basic/atomic outcomes. Preferences are typically operationalized as "revealed preferences", i.e. reliable choice when presented with two (or more) alternatives. Thus observed revealed preferences are (in this ontos) taken to be (along with probabilistic beliefs) the latents driving rational behavior.

This, however, is not quite how the term "preference" is being used in daily language. I conjecture that we invoke the notion of "preference" (primarily?) when we witness an agent-y/mind-y entity behaving in a consistent/repetitive way in similar choiciness context, despite not having to be/do so, i.e. being underconstrained.

There's a bit to unpack so let's unpack it.

The «context» (/situation) the agent finds itself in imposes some "constrains" on the "actions" available to the agent. It limits the agent's "degrees of freedom". When we «interpret» a situation as an "«instance» of this pattern", this influences our expectations of what the agent may do. Sometimes the constraints are complete: there is only one thing the agent can do. Otherwise, the agent is "underconstrainted": there is more than one thing may do, for all we know. I will use the term "choiciness" or "choicy contexts" to refer to such situations, characterized by there being more than one «action» available to the agent, from our standpoint. If we observe the agent repeatedly in "contexts" that we recognize as «sufficiently similar» («instances» of the same «type»?) and the agent's behavior is «sufficiently similar» across these contexts, we ["explain"]/["make sense" of] it using the notion of preference. The agent does X repeatedly because it prefers X (to the alternatives available in the «context»).

Importantly, by "constraints" I don't just mean "causal constraints" like having one's arm tied to the body, impossibilizing movement. I also mean "telic constraints" like being encultured so as to behave a particular way in a particular context even though technically other ways of behaving are available and if one were offered a lot of money or something, they would breach this social norm or habit of behavior.2 This means that there is an obvious/natural «default». Preferences, I conjecture, are contingent latents supposed to explain consistent/repetitive dynamic in contexts where «defaults» are absent. What "counts" as a constraint depends on the (type of) thing/agent. You can constrain a normal human by putting them in a sealed chamber but it's not going to work for a Pillar Man.

In the previous paragraph I mentioned the case where the agent "violates" its «default» when doing so grants it a sufficiently high amount of money (or whatever other thing it prefers). How should we think about it? I lean towards seeing the money offer as transforming a non-«choicy»/«default»-y context into a choicy context in which the agent has/expresses greater preference for the money offerred than for (what was previously) the «default».

Another important thing: repeated/consistent observations of reliably repetitive/consistent behavior in a choicy context (in a class of choicy contexts?) make us infer a preference. This preference then becomes incorporated into our machinery for modeling this particular agent (and also possibly other agents similar to it) and thus becomes usable as "just another constraint" (probably a telic constraint). But we don't treat it as "just another constraint". It is a preferrence. We keep track of it as being an intrinsic property contingent/specific to a particular agent (relative to a relevant reference class). That preference "could be otherwise". Still, it expands our notion of «default» specific to this agent. On the other hand, the distinction between preferences and telic constraints is blurry.

If the agent then violates our notion of «default» specific to it, we are surprised and inclined to explain it in terms of some additional constraints (intrinsic to the agent or extrinsic/contextual) or mistaken inference about the preference in question or a change in its preferences, etc.

Having preliminarily discussed preferences, I will use them as a starting point for discussing features of the V-cluster in general.

V-in-general

Ontos-dependence

This entire picture/analysis of preference rests on the assumption that the observer/modeller uses an ontology that smoothly accommodates the concept of "preference" (one might even say it sweeps it under the rug). First, it has a way of discretizing the «flux of the world» into «contexts» (that are a kind of «thing»), discretizing the underconstrained «action/option» space into specifiic "actions/options", and comparing the elements of these discretized spaces as "sufficiently similar". One way to make some progress on making sense of this is the Natural Abstraction Hypothesis but despite the ongoing mathematization of NAH, it still seems pretty skeletal.

Involved in temporal discretization is also the issue of the scale of "temporal chunking", i.e. how long are the "contexts"? Intuitively, "choosing" and "option" connote a kind of instantaneity/all-or-nothing-ness which fits well with the notion of "preference" adopted by axiomatic rationality. Even in the case of dynamic choice, there is a sequence of "contexts" that the agent is faced with and needs to choose one of the available options in each of them. The longer are the "contexts" as chunks of time, the more thinking about the agent's behavior in terms of discrete-choice and axiomatic-rationality-preference feel like a stretch and other V-concepts, such as wanting or goal-directedness, seem more appropriate.

The point about the need to discretize the option space is not quite right. An agent can reliably optimize for some function across sufficiently similar choicy "contexts" and we might still describe this as a preference.

This is also related to the fact that we're not trying to predict the behavior with maximal precision only up to some relevance or watnot. The level of precision/detailedness is very context-dependent. Sometimes we use the term "preference" to refer to quite precise behaviors, e.g. choosing black theme instead of white theme on LessWrong when confronted with that pair of choices. Other times we use the term "preference" to summarize patterns of behavior, e.g. "he wants to warm up" summarizes "he's getting a blanket, he's moving closer to the fire, he's getting hot chocolate". This also highlights that repeated choices of A over B don't mean «basic preference» for A over B but typically for something that A forward-chains to over whatever B can forward-chains to (since the things we directly control/choose typically are not the things we care about).

More broadly, since these explanations of agent-y behavior are dependent on the observer/modeller's ontos, they are influenced by its «inductive biases». In particular, a bias towards some kind of parsimony. For example, postulating one preference to explain a lot of the human's behavior is preferred (by the observer/modeller) to many disparate preferences explaining various disconnected particulars of the human's behavior. Parsimony (~[short description length]?) is not universal; it's ontos-dependent (although there may be some common selection pressures drastically narrowing the range of viable ontoses in our universe, by something like natural abstraction and/or selection theorems).3 At the same time, it's not the only thing. It's intuitively a stretch to say that humans "just" "want"/"prefer" more [whatever it is in their brains that makes them do goal-directed things]. This leads to another inductive bias: There are some typical things for humans (or more generally, a particular class of minds/things/agents) to prefer/want (have a V that [refers to]/[is directed at] them). What are these things? Perhaps there is a way to make sense of them in terms of parsimony/[short description length] from the perspective of the ontos the entity being modelled (at which point we stop treating the entity as a black box), thingness-y things, natural abstractions, or Markov blankets/causal blankets/«Boundaries»,4 or something like that.

The dual role of V

(Maybe also relevant: Gemini modeling.)

Until now, I discussed V-concepts exclusively in one thing modeling/predicting/[trying to understand] the behavior of another thing. As I will lay out in the subsequent paragraphs, this (to me) seems to be the most fundamental role of V-concepts, from which all other roles are derived. Still, the V-concepts also seem to have another aspect that seems (FWIW) qualitatively distinct from the previous one, even if grounded in it.

I.e. we use V-concepts not only to «comprehend» behavior of the other but also to «comprehend» our own behavior. As a special case, we use them to "justify"/"explain" our behavior to others, to help them «comprehend» our behavior, or maybe even ensure that they «comprehend» our behavior the way we would like them to (e.g. because we want their «comprehension» to «agree» with our «comprehension»).

(We might call these two aspects/use-cases "retrospective/explanatory" and "prospective/goal-oriented". If Alice's retrospective/explanatory understanding of Bob's is accurate and Bob's prospective/goal-oriented understanding of Bob-self is accurate, these two should align.)

In what situations do we (feel the) need to help somebody else «comprehend» our behavior?

One class of such situations are those in which the other perceives our behavior as surprising (incongruent with the constraints that they perceived to be imposed on our behavior) or underconstrained (and therefore requiring a constraint complementation) AND we prefer them to have that information. If we don't deliver this information (what kind of V-thing guided our behavior), the other may mis-infer our preferences. We think we know our preferences better than the Other and are therefore in some cases inclined to do the work of inferring our preferences for the Other so as to prevent possible misinference/misinterpretation. Importantly, this assumes that we have acces to the ground truth whereas the Other doesn't; or at least that we have better/closer access to the ground truth (something something data processing inequality). This is likely the case [on average]/typically but sometimes the Other infers our preferences (and aspects of our cognition more generally) better than ourselves.

Under all this, moreover, there lies the assumption that there is some ground truth. Is this the (unique) ground truth or a (not necessarily unique) ground truth? It seems like we intuitively assume the V-ground-truth to be unique. However, one V-ground-truth can be consistent with multiple explanations. For one thing, a V-explanation can be given at different "points"/"levels" along the instrumental-terminal spectrum (to the extent that concept of instrumentality/terminality is applicable here). For another thing, one can choose to emphasize different aspects of the V or summarize/compress it in different ways. This communicative aspect functions both to communicate between different agents and for one agent to communicate between itself in different contexts or at different points in time. This looks like a special case of formulating an instrumental strategy.

To the extent that the V-ground-truth is affected by the process of inferring/[trying to «comprehend»] it, it has a hyperstitious character. For example, according to the FIAT hypothesis, «coherence»/goal-«directedness» fictitiously [inferred from]/[ascribed to] one's past behavior is an important mechanisms by which humans acquire their V-things. There are also other non-FIAT mechanisms by which (at least communicatively/«reflectively» endorsed) values are acquired by re-"interpretating" one's past behavior although the lines here may be blurry in that re-«interpretation» may be (partly) driven by noticed in-«coherence» and one might then re-«interpret» one's V-things in some way that marginally increases (perceived) «coherence». (This opens two further directions of investigation: what is (re-)«interpretation» and what is «coherence»?)

(FIAT may also be seen as possibly hallucinated decompression, (implicitly) assuming that past behavior is a compression of something even though this might not be actually the case.)

Another (intra-?)communicative purpose of V-concepts is something like goal-factoring. (This is not actually circular because a system might have a V-thing (e.g. goal) without having a V-concept. A bacterium moving up the sugar gradient is an example.) You need to have a concept of the thing you're trying to factor in order to factor it realiably. (Unless you can do some kind of enactive goal-factoring but it's not clear to me whether it's possible and I'm overall confused about (radical-ish?) enactivism in general.)

Sometimes V-talk occurs in the context of "preferring/valueing/v-ing some X even though one doesn't in any other way take actions in the direction of X (getting more of X, in expectation)". "Akrasia" is one word used for some cases of this. More generally, we might call it "a preference that is not a revealed preference" (in the economics sense). I'm not sure how to think about this but here are some possibilities. (These possibilities are not mutually exclusive. There may be (likely are?) many mechanisms underlying one appearance.)

  • Confusion: The agent doesn't actually know what they prefer.
  • Signalling (deception?): The agent says that they prefer X [even though they don't]/[regardless of whether] they actually prefer X in order to signal allegiance to the group, some virtue, or whatever.
    • Such half-assed-ness in the form of lack of behavioral manifestation may diminish credibility. On the other hand, half-assed-ness may not diminish credibility if it is in the context where e.g. it's super-rare and/or costly for non-members to signal this kind of stuff. Somebody may not actually believe that "we should kill all the rich people" and they may not take any object-level actions implicated by this belief either. However, by signalling this, they put off everybody else, which shows that they are willing to (symbolically?) sacrifice quite a lot for the group.
    • Related: Crony Beliefs by Kevin Simler.
    • This certainly doesn't cover all the cases but it covers some.
      • Moreover, it seems plausible that this can turn into a non-akratic/"real" preference (e.g. via instrumentally useful pre-commitment or self-modification), so if someone eventually acts on what was expected to be a crony preference, is not necessarily decisive evidence against its croniness (at least until the moment of taking action).
  • Bounded rationality: The preference is there but the agent's actuation/optimization power are very limited. E.g. the agent may have some non entrenched behavioral patterns that it simply can't refactor on its own.
  • Subagents: There is a subagent (a want? a thread?) that has the preference X but is not "permited" (?) to act on it.
    • There may be a parliament of wants that each has access to a veto and this agent is "too vetoed".
      • If this is on the right track, I expect the right picture to be quite a bit more messy. For example: (1) different subagents are activated to different extents in different contexts; (2) it's more complicated than unilateralist vetoing; (3) the number of subagents is not static; (4) the boundaries between subagents may not be determinate and they may be able to cooperate in some way.
        • Relatedly to (3) and (4), re-«interpretation» may be a major mechanism for changing the effective subagenty structure.
    • This begs the question of why is speech access less veto-able. Relatedly:
      • This may also be about the disconnect between behavior and «reflectively approved» V-things.
      • It relates to the question of why do certain structures [get access to]/[appear in] in (conscious? verbally accessible/enabled) awareness rather than others.
    • Maybe relevant: Knightian Decision Theory, action gating mechanisms in basal ganglia.
  • Frustrated wanting (h/t Lauren)
    • Some version of trying and failing where failure occurs at a very early stage.

We also use V-terms to coordinate with others, negotiate, trade, etc. We're not always truthful about it. We can communicate our preference falsely because it gets us what we want more effectively than communicating our preference truthfully.

I see at least two mildly interesting things about this (use) case.

  1. Here, the V-concept is used as an interface/protocol by which two agents can communicate.
  2. When we're assessing/predicting/witnessing the agent's behavior in this kind of underconstrained/choicy context, we use the concept of "preference" to explain its usage of the concept of "preference".

That's it for the discussion of V-in-general. The following sections contain mostly brief analyses of V-terms other than "preference".

Goal, purpose, objective

These denote a specific kind of thing (state? event? situation? proposition in Jeffrey-Bolker?) that the system is «trying» to move (in)to.

I've read Deconfusing Goal-Directedness but it seems mostly like a partial elaboration/explication of our confusion about goal-directedness. The most promising things I see there is something like:

A central aspect of goal-directed (/purposeful/objective-driven) behavior is that it can be described as aiming for something specific despite (relatively) strong distributional shifts and perturbations to the trajectory (according to such-and-such ontos).

But this is about describability. I want something stronger, something that talks about "how the system is", rather than "what the system appears to be like" to an observer with such-and-such ontos (though I don't necessarily want to eliminate the ontos as a parameter in this description). I want to go beyond the intentional stance.

Value

We seem to be using the V-word "value" in two different senses.

  1. Abstract
    1. Abstract summary of (some subset of) an agent's V-stuff, typically «reflectively endorsed», though not always.
  2. Evaluative
    1. Evaluating something as positive/negative. Not necessarily translating into behavior (though typically it does translate into behavior). (Probably related to liking vs wanting.)

Sometimes we talk about "choosing one's values". This seems fruitful to think about in terms of underconstraint/choiciness in what one cares about.

Like

The term "like" highlights the evaluative aspect of "value". It may not, though typically does, overlap with wanting.

Care

"Caring" seems to denote a kind of «directedness» but (intuitively) of a more general kind than goal-directedness. For example, it feels like a stretch to say that "I care about person X" means "I have a goal that person X is in a good state, has high welfare, their preferences are satisfied, their life is going well or whatever". The target of this «directedness» seems more blurry, nebulous, left-to-be-specified, e.g. by the person being a target of my caring. At the same time, I might really care about performing well on some narrow task, in which case we have both goal-directedness and caring. On the other hand, it seems to me that when we talk about humans "caring" about some narrow goal G, this is usually in the context of G serving some other nebulous purpose. For example, when I hear "James really cares about maximizing his work performance", I'm inclined to expect something like:

  • a) James wants to be seen as a good employer.
  • b) James cares about the prosperity of his company (maybe because it has some stated mission that James cares about).
  • c) James cares about earning a lot of money (probably because of its instrumental value for whatever are the typical human things James cares about).
  • d) James imposes on himself some perfectionist/optimizer-y standards.

All of which are generally more nebulous than "maxing out one's work performance".

"Caring" definitely has an evaluative aspect but we also expect people to act in a way that is "coherent" with their caring (to the extent that one is capable of that action), moreso than when we talk about "mere" liking or "valuing" in the evaluative sense.

Try, aim, pursue

These belong to the V-cluster although perhaps somewhat more to the perifery than to the center. The (more) central V-terms denote things (V-elements) that are capable of steering behavior. That steering may be successful or unsuccessful. "Trying" has two (context-dependent) senses/interpretations. One refers to events where a V-element has interfaced with the [other elements]/[rest of the world] in a way typical of cases in which they subsequently «succeed». The V-element served its "function" in the broader system/context of [making some X happen] where "X" stands for "the thing being tried". The other sense/interpretation refers to a subset of the former where X didn't happen, despite the trying; i.e. lack of «success». (We might explicate the meanings to assign the former meaning to "aim/"pursue" and the latter meaning to "try".)

How can one figure out what is the function of a V-element in the X-structure, with "X" standing for e.g. "build an AI that satisfies such-and-such specification criteria"?

(The following are speculations inspired by Deacon.)

One hope is that this kind of task/goal structure re-uses some kind of template. Suppose that there is a common way that goal-directed processes running on a mind arrange themselves with a «reference/pointer» to some [thing X in the world, (possibly) not yet existing] such that the process ceases when X is realized (unless the processes lose their coherence for whatever reason, including [negative feedback]/interference from unforeseen/unappreciated difficulty of X). It would then be somewhat easier to see that a mind is "aiming"/"pursuing" if we could look into its internals and decode them sufficiently well (which is not a given). Also, it hinges on the concept of "reference/pointer" and "criterion of ceasure (malfunction)" (that distinguishes aiming/pursuing from trying, in the senses given in the first paragraph of the current section), which remain to be analyzed next.

Aside: trying, expectation, wireheading, ethical injunctions

Do. Or do not. There is no try.

~ Yoda

(The following is a somewhat anecdotally-and-introspectively-supported speculation that might well be proven wrong by some existing high-quality body of psychological evidence that I'm not aware of yet.)

Humans often don't actually aim/pursue reliably/robustly. They do stuff that (in the judgment of relevant observers, often including the doer themself) would "count" as a sufficiently good try. This is a meaing of Yoda's quote and Trying to Try. Perhaps if humans adopted the previously proposed explications of aiming/pursuing vs trying, telling them to "aim" rather than "try" would be marginally more effective but probably not that much more effective. In a sense, humans cannot «control» their achievement of goals, they can only «control» (the things that influence) their «expectations» of their achievement of goals. To the extent that these «expectations» are miscalibrated, especially in assigning a too high weight to factors that are easy to influence, this opens the possibility for wireheading.

Relevant (in a way to be elaborated): Ethical Injunctions, The Thing and the Symbolic Representation of The Thing, The Good Try Rule.

Want, desire, drive, motive

These four seem to denote intrinsic-to-agent latents that cause it to behave "directedly". Like "preference" or "goal", they are trying to be serve the role of the explanans of a coherently directed behavior.

The main difference that I see between them is that "wanting" seems to denote more of an «ownership by/integration with» the wanter. The wanter acts towards X because they want X but that wanting is an integral part of the wanter. Desire, drive, and motive, on the other hand, seem to introduce a possibility of "alienation", they make the wanter do stuff, perhaps somewhat independently of wanting? Moreover, desire and drive feel more internal to the wanter, whereas the motive is possibly more externally imposed, like a «reason» to do/pursue something.

Intention

Intentions seem more [potentially less actualized] than desires, motives, drives. They seem more latent, "in the realm of potentiality/possibility, rather than actuality".

Relevant: Possibilizing vs. actualizing.

Peter Godfrey-Smith: Intentions as psychological states mediating action constancies (i.e., I want X to happen, less important how I make X happen).

Further threads

  • «flux of the world» and how it's chunked into «contexts»
  • «thing»
  • «action», «option»
  • «constrain», «constraint», «degrees of freedom», «could be/do otherwise», «free will»
  • «sufficiently similar» «instances» of a «type»
  • «default»
  • «basic preference» / «basic V-*»
  • «comprehend», «explain», «understand», «make sense of»; «agreement» between two «comprehensions»
  • «coherence», «directedness>
  • «reference», «pointer», «representation», «interpretation» → Intentionality
  • «function»
  • «reflection» / «reflective» «approval»/«endorsement»
  • «evaluation»
  • «succeed»
  • «control»
  • «expectation»
  • «ownership by/integration with» vs «alienation»
  • «reason»
  • «inductive bias»
  • «boundaries», «individuation»

Also:

  • Ambition
  • What distinguishes V-things from other things?

What distinguishes V-things from other things?

V-things are used to complete constraining of behaviors in choicy contexts. (At least in our explanations/models/etc.) (For now I'm thinking of single-situational choiciness.)

Naively speaking, V-things are not the only "internal" behavior [co-determiners/constraint completers]. There's shit like epilepsy, impulses/OCD, addictions and so on. Depending on (the choice of) the ontos/boundary of InternalMindSpace, different things count as internal co-determiners (in the relevant/[locally assumed] sense).

  • Hypothesis: V-things are those (types of) internal/intrinsic behavior co-determiners that are potentially possible to negotiate/reason/dispute with?
    • Hypothesis: V-things are disputed. The rest is modulated.
    • Hypothesis: V-things are reason-apt: potentially reasonable or unreasonable, rational or ir-rational. The rest is a-rational.
      • But what then is «reason(ableness)»?
  • One aspect of reasonableness (in this context) is potential [temporal horizon flexibility], like the ability to sacrifice short-term V-fulfillment for longer-term V-fulfillment (maybe aggregate, after appropriate discounting etc).

What other properties may be uniquely characteristic of V-things?

  • Unitedness in a cabal?
    • This could be subsumed by the prior hypotheses. In order to work within a cabal, a V-thing needs to be (sufficiently) reason-apt.
      • cosmopolitan-Leviathan
  • FIAT?
    • In order to impute coherency to some of one's past behavior, one needs to view this behavior as coherence-apt and thus reason-apt.
      • Is it actually the case? The term "coherence" seems to be load-bearing.
      • I don't interpret my impulses as [pursuing goals that I may want to adopt]. Or do I? Sometimes they may be taken as indicating something important to me that I've been suppressing.
        • So then a non-V-thing is re-interpreted as a V-thing and then FIAT-ed.
  • Process of discovery/construction? (Liberalism? h/t TJ)
    • Scylla and Charibdis. @smithOriginObjects1996

V-thingies and choiciness are tied to our intuitive notions of responsibility (including its various derivations/elaborations like gullibility, blameworthiness, praiseworthiness, etc).

When a person voices that they want/prefer/value something or when they are perceived as wanting/preferring/valuing something, they don't seem to be making a choice (in the paradigmatic sense of the word, whatever it means), nor do they seem to be "enslaved" by their V-thingies. One is "centrally" enslaved when one is addicted to something, "less centrally" when they want X but they [don't want to want X]. There seems to be a degree of non-central, weird choiciness in that one «kinda chooses» to follow/obey/execute this particular combination/ensemble of V-thingies.

Fast reactions, reflexes, and habits are not central examples of V-thingies either. It would seem like an important part of V-thingies is [the capacity to function as a «reason-input» to T2 that can steer behavior/override default behavior].

Footnotes

  1. I'm primarily discussing English here for obvious reasons. In this manner, English doesn't seem to deviate from other Indo-European languages, at least those that I know of, although they belong to the same moder-Western (WEIRD) cultural bubble. The extent to which the conceptual scopes discussed here are parallelled in other languages is an open question. I'm especially interested in how these things are spoken about in mind-opacity cultures, such as Yasawa of Fiji.

  2. In a sense, telic constraints "translate to"/"cache out in terms of" (complicated patterns of low-level) causal constraints. However, it's the perspective of telic constraints, not of (complicated patterns of low-level) causal constraints that we use in modeling an agent's behavior. There's a longer argument on more fundamental-ish relevance of telic constraints but beyond the scope of the current writeup.

  3. It seems possible that all inductive biases might be unified under some useful notion of parsimony but it also seems possible that we would want a richer conceptual toolbox giving parsimony a narrower denotation.

  4. Various blanket/boundary formalism may be useful for constraining what kinds of things the agent may plausibly prefer/want/[care about] in that they may specify some carving of the world that the (generalized) domain of preference needs to align itself with.