Expected Utility All the Way Up?
[WIP]
Some theories of (rational) agency postulate that the most primordial/basic normativity/imperative is not about doing anything in particular (acting) but about constituting oneself as an agent, pulling oneself together into a single unified whole, so that there is something over and above the individual parts. An agent is something more than a disorganized "mere heap" of its components. It is an acting entity the many parts of which work in synchrony towards a common goal. Therefore, to act at all, one needs to coordinate the many parts of oneself first.
However, one can take a dual perspective and think of a concrete action as an instrumental means. Or, more specifically, a specific end (a thing to be done) is being selected1 to serve as a coordination point for the proto-agent's (prospective agent's) parts.
To rephrase, there is some pressure operating on the proto-agent, a pressure that pushes it in the direction of unifying into a proper agent. In a similar way that (what we call life) arises as a solution to the problem of how to efficiently dissipate energy,2 an end arises as a solution to the problem of how to coordinate the subagentic parts. To reach for yet another metaphor, the end is a conductor to the orchestra of parts. By coordinating around this end, the parts turn into means in service of this end.
Examples of this view and sources of inspiration
This is in line with Neo-Kantian philosophy of rational action or at least its interpretation by Christine Korsgaard: to act at all, the agent needs to be an autonomous and efficacious entity, i.e. a proper agent, and the way that an agent turns itself into agent is by acting.
Peter Godfrey-Smith writes in Other Minds on two views of the original/primal function of animals' nervous systems. On one view (the more popular one), nervous systems evolved primarily for action. On another view, they evolved for action shaping, coordination, with some ability to sense and act in a coordinated manner coming for free as a side effect of this. Fred Keijzer suggests one specific model which he dubs the Skin Brain Thesis (SBT), according to which (proto-)nervous tissue started as an adaptive external layer of cells (skin brain) moving in a coordinated manner based "only" on their communication with each other (in particular, their relative placement), rather than any sensing from the outside per se. At that point, the only "sense" this tissue had access to was a very primitive form of proprioception. However, that primitive form of proprioception was sufficient to enable the development of a primitive form of touch because (subtle and not so subtle) mechanical disturbances of the spatial configuration of that tissue's cells naturally led to changes in what theses cells perceived from each other. The need to increase efficiency, precision, and speed of communication across the skin brain (especially between the cells placed further away from each other) led to the development of long axon/dendrite-like appendages in some specialized cells, making them proto-neurons.
John Wentworth and David Lorell write that Instrumental Goals Are A Different And Friendlier Kind Of Thing Than Terminal Goals which made me think "Ok, so can we have an instrumental goal-centric view of agency, and if so, what role would terminal goals play in such a view?".
In some episode of Minds Almost Meeting, Robin Hanson3 said something along the lines of "I don't care what our descendants are going to optimize for. I want it to be the case that whatever they optimize for, they do something interesting.".
Davies and Lineweaver propose the atavism hypothesis of cancer. They point out that the phenomenon of cancer seems … kind of too lucky. In particular, a small number of mutations is sufficient to enable a whole host of cancerour adapatations, a number that is deleterious to the normal/healthy mode of functioning, but apparently cancer tumors can sustain a much larger number of mutations while still remaining fit as cancers. Their proposed explanation is that what we call cancer is an atavistic reversion to a previous mode of living: Metazoa 1.0. Primitive (proto-)animals, they say, were very much like modern cancers but a relatively small overlay of genetic mutations and epigenetic regulation mechanisms allowed them to "ascend" to a higher level of coordination/organization, what we call Eumetazoa or Metazoa 2.0. It is tempting to consider the parallels between this phenomenon and some phenomena in the lives of human collectives. E.O. Wilson said
The real problem of humanity is the following: We have Paleolithic emotions, medieval institutions and godlike technology. And it is terrifically dangerous, and it is now approaching a point of crisis overall.
This is not strictly true. Human (genetic) evolution didn't stop with the neolithic transition, so the genes that co-determine the brains of modern humans are somewhat different from those that co-determined the brains of their hunter-gatherer ancestors. Similarly, we don't literally have medieval institutions, for better, and/or worse. But the point may stand: a lot of what determines intelligent behavior of modern human collectives is adaptations/factors that are less appropriate now than they were N years ago, overlain by a small number of "corrections" riding on top of them, and when those "corrections" break down, a cancer-like chaos of reversal to our previous modes of living risks ensuing.