Robust Constraint

[Work in progress.]

Robust constraint

Here, I'm introducing a notion of (parametrized-)robust constraint which I expect to be a good basis for a notion of optimization, conditional on it even turning out that we want to retain the notion of optimization.

Basics

$\mathcal S$ - A state space (a set) of the system being optimized.
$\mathcal C$ - A set of possible constraints that can be put on the system.
$f:\mathcal P(\mathcal C)\to\mathcal P(\mathcal S)$ - A function mapping a set of constraints $C\subset\mathcal C$ to a set of system states compatible with those constraints $f(C)\subset\mathcal S$ .
$\tau:\mathcal P(\mathcal S)\to\mathcal P(\mathcal S)$ - A possibilistic transition function.
- We abuse the notation and allow ourselves to write $\tau(s)=\tau(\set s)$ for $s\in\mathcal S$ .
$f^{-1}:\mathcal P(\mathcal S)\to\mathcal P(\mathcal P(\mathcal C))$ - The inverse image of $f$ .
$f^*:\mathcal P(\mathcal S)\to\mathcal P(\mathcal C)$ - A pseudoinverse of $f$ , obtained by $f^*(S):=\bigcup\set{X\in f^{-1}(S)}$ .
- In general, $C\subset f^*(f(C))$ but not necesesarily $C=f^*(f(C))$ . If equality is satisfied, then we say that the constraint sets and (underlying) state sets are (uniquely) identifiable. This is convenient, because it allows us to define a higher-level transition function $\tau_\mathcal C:\mathcal P(\mathcal C)\to\mathcal P(\mathcal C)$ , as defined by $\tau_\mathcal C:=f^*\circ\tau\circ f$ .
$\Phi:\mathcal S\to P$ - A predicate on $\mathcal S$ describing the degree of optimization of a state $s\in\mathcal S$ , where $P$ is a lower semi-lattice.
- We abuse notation and write $\Phi(S)=\set{\Phi(s):s\in S}$ for $S\subset\mathcal S$ .

This suffices to define $(t,p)$ -robust constraint.

The set of constraints $C\subset\mathcal C$ $(t,p)$ -robustly constrains $\mathcal S$ if $\inf(\Phi\circ\tau^t\circ f)(C)\ge p$ (for some $t\in\mathbb N$ and some $p\in P$ ).

This means that the set of contraints that we imposed on the system to keep it within the target region that is at least as good as $p$ for at least some desired time $t$ .

For a particular case of binary constraint $\Phi:\mathcal S\to\set{0,1}$ , it is arguably rather obvious that we want to use $1\in\set{0,1}$ , and therefore we can just say:

The set of constraints $C\subset\mathcal C$ $t$ -robustly constrains $\mathcal S$ if $(\Phi\circ\tau^t\circ f)(C)=\set1$ (for some $t\in\mathbb N$ ).

This means that the set of contraints that we imposed on the system to keep it within the target region $\Phi^{-1}(1)\subset\mathcal S$ for at least some desired time $t$ .

(Note that the use of a more general poset $P$ is equivalent to the uses of a family of binary $\Phi_p:\mathcal S\to\set{0,1}$ 's, parametrized by $p\in P$ .)

Extensions and variants

This is just a sketch for temporally robust constraint with a possibilistic transition map $\tau:\mathcal S\to\mathcal P(\mathcal S)$ that expresses our state of knowledge about the environment.

We could also talk about it in terms of progression on the level of constraints alone: $(\Phi\circ f\circ\tau_\mathcal C^t)(C)=\set1$ .

Possible extensions of this proto-framework:

Extend the transition map to be probabilistic ( $\tau:\mathcal S\to\Delta\mathcal S$ ) and/or involve some interactions with the environment ( $\tau:\mathcal{S×E}\to\mathcal S$ ).
Impose some additional application-specific structure on the spaces that allows you to prove something in those specific applications.
Extend to multi-level setups, e.g. hierarchies of states constraints $\mathcal S,\mathcal S',\mathcal S''$ , etc.
Extend to robust constraint across more dimensions than time. Perhaps there is a way in which we can talk about robust constraint without (explicitly) referencing time at all, e.g. a node in the network of interactions is constrained in such a way that its neighborhood also becomes constrained in some transitive way.

Relationship to optimization

Why is this something we might want specifically for the purpose of talking about optimization?

Intuitively, all optimization is robust optimization. To optimize something in the world according to some desideratum is ensure that this desideratum is satisfied and that this desideratum's satisfaction won't be perturbed by some happenings in the world. If you're optimizing a piece of rock, you probably don't want this rock to dissolve into nothingness 100 miliseconds after you finished your deed.

But what if you do want that? Or at least, what if you don't mind that? What if your plan is to optimize a piece of rock and only care that it stays optimized for 100 miliseconds?

Then there is still a sense in which you need to optimize something really robustly to perturbations and that is your policy to optimize that piece of rock. You want to be a robust optimizer (or to robustly implement an optimizing policy/plan) across the time span that it takes you to optimize that piece of rock.

Therefore, the concept of robust constraint is implicit/latent both in (1) the notion of something being optimized for something; and (2) the notion of optimizing for something.

I resorted to talking about robust constraint instead of optimization. This partly evades the degenerate representation objection: "You can always say that this random thing there is optimized to be doing whatever it is actually doing.". This feels like a much weaker objection if we talk about constraint, not optimization, because constraint doesn't optimize telicity/purposefulness/goals/normativity.¹

I am inclined to speculate that the central discussions in the vicinity of AI risk involving the concept of optimization are trying to point at phenomena that are best described by a two-level model of robust constraints.

Let the notation be as before, except now instead of a single constraint space $\mathcal C$ , we also have another constraint space $\mathcal D$ that is treating $\mathcal C$ as its state space.

[WIP: to be continued…]

Other ideas for what optimization involves

I don't necessarily expect my takes on other approaches in this direction to be very convincing. I'm largely following my not quite legible sense of what directions of thinking are promising. Here I'm mostly explaining the legible parts of my reservations.

Concentration of probability in a previous unlikely region of the probability space

There is a LessWrong-canonical notion of optimization (due, I think, to Eliezer Yudkowsky), according to which:

Optimization is any kind of process that systematically comes up with solutions that are better than the solution used before. More technically, this kind of process moves the world into a specific and unexpected set of states by searching through a large search space, hitting small and low probability targets. When this process is gradually guided by some agent into some specific state, through searching specific targets, we can say it prefers that state.

In Optimization, Eliezer writes:

Similarly with the problem of planning, which also involves hitting tiny targets in a huge search space. Consider the number of possible legal chess moves versus the number of winning moves.

Which suggests one theoretical way to measure optimization - to quantify the power of a mind or mindlike process:

Put a measure on the state space - if it's discrete, you can just count. Then collect all the states which are equal to or greater than the observed outcome, in that optimization process's implicit or explicit preference ordering. Sum or integrate over the total size of all such states. Divide by the total volume of the state space. This gives you the power of the optimization process measured in terms of the improbabilities that it can produce - that is, improbability of a random selection producing an equally good result, relative to a measure and a preference ordering.

If you prefer, you can take the reciprocal of this improbability (1/1000 becomes 1000) and then take the logarithm base 2. This gives you the power of the optimization process in bits. An optimizer that exerts 20 bits of power can hit a target that's one in a million.

When I think you're a powerful intelligence, and I think I know something about your preferences, then I'll predict that you'll steer reality into regions that are higher in your preference ordering. The more intelligent I believe you are, the more probability I'll concentrate into outcomes that I believe are higher in your preference ordering.

[…]

If you want to measure the intelligence of a system, I would suggest measuring its optimization power as before, but then dividing by the resources used. Or you might measure the degree of prior cognitive optimization required to achieve the same result using equal or fewer resources. Intelligence, in other words, is efficient optimization. This is why I say that evolution is stupid by human standards, even though we can't yet build a butterfly: Human engineers use vastly less time/material resources than a global ecosystem of millions of species proceeding through biological evolution, and so we're catching up fast.

On first glance, this does feel somewhat compeling. It feels like there is some there there. We're assuming some probability or measure over the states of the system. An agent's optimization power is its ability to convert resources into concentration of that probability/measure into regions higher in its preference ordering. However, there's a lot of free parameters.

Different agents have different preference orderings.
- What is the scope of the preference ordering, spatiotemporally speaking?
Where should we get the ontology for the system and its probability/measure? If the system includes the agent, then all its impacts are already priced in. If the system doesn't include the agent, then we're applying some kind of do-operator. Presumably we are some limited/bounded observer that needs to resort to action under uncertainty.
The concept of optimization power relies on some unified, scalar notion of resources.
Presumably, for this concept to be useful, it should be somewhat repeatable across different contexts, transferrable.

The ground of optimization

This is way too lenient. I reject notions of optimization that include a ball rolling downhill and settilng at the minimum point (absent good theoretical justifications or evidence of the notion's usefulness).

Although it is plausibly a ground on basis that might serve as a foundation for talking about normativity. ↩

Robust Constraint