Watch only the Road: Activity vs. Computation

One criticism that is sometimes raised (e.g. by Maudlin, by Marchal) against the idea that a valid criterion for implementation of a computation can exist is that what consciousness exists should not depend on "what would happen" in counterfactual situations. Yet, counterfactuals are needed to define the causal relationships that distinguish a run of a computation from a mere sequence of states, and a such mere sequence could be easily found in random or repetitive data and should not be conscious.

This criticism gains intuitive force - and begins to seem plausible - when the counterfactual situation would be a complicated one.

For example, consider the following system:

I1,I2 are inputs
If I1 = I2 then Black Box 1 is activated, making state A = a; otherwise A = 0
If I1 does not = I2 then Black Box 2 is activated, making state B = b (b is not equal to a); otherwise B = 0
Output O1 = A + B
The value of O1 can then be used for further steps in a computation, e.g.
O2 = TRUE if O1 = a, else O2 = FALSE

This system implements the computation (I1, I2, O2) ==> (I1 , I2 , (I1=I2))

Now assume that I1 = I2. Black Box 1 was activated, and in the end O1 = a and O2 = TRUE.

We also know that if I1 had not been = to I2, then Black Box 2 would have been activated, and in the end the outputs would have been O1 = b and O2 = FALSE.

The problem is that Black Box 2 could contain a very complicated apparatus, such as a Rube Goldberg machine. If we didn't already know, it would be very difficult to tell what the effect of activating the box would actually be, without activating it to find out. Besides that, the box might work by sending a signal to a distant planet which is luckily inhabited, waiting many years while the aliens' culture evolves to the point that they can understand the signal, and then receiving back the message "b" from the aliens.

Given that in the actual scenario, Black Box 2 was never even activated, can whether the computation was implemented or not (and thus, if computationalism is assumed, perhaps whether or not an AI was conscious) really depend on such distant and complicated details? What if the aliens would have transmitted the message "a" instead of "b"?

By definition, whether the computation was implemented or not does indeed depend on those global counterfactual details. However, it does not seem plausible that whether an AI was conscious or not (and if so, of what) could have depended on those complicated counterfactual details. Must we then abandon computationalism?

To keep computationalism we need to keep causality, and thus counterfactuals, but avoid complicated chains of counterfactual events. There is a way to do it, and it is by considering equivalence classes of computations with respect to consciousness. If a given run of a computation would have given rise to a conscious observation, then it must be the case that a run of a different computation which had both the same sequence of actual states and the same causal relationships between those states would have given rise to the same consciousness, even if that computation would have behaved differently if the initial state had been different. I will call such runs, together with the causal relationships, a computational "road".

But how do we define those causal relationships, if not by counterfactual relationships?

If we consider the underlying system, there is no real problem if the system is allowed to evolve for a single 'time step'. I call the one-step considerations 'local causality'. The problem arises when complicated chains of events - global causality - would have come into play.

Causality must have the following characteristics:
If A causes B (written as A ==> B), then:

If A occurs, then B _must_ occur.

If A _does not_ occur, then B _might_ occur. In order to find out, we would need to know the laws of the underlying system, which will tell us what independent variables (of the underlying system or of a valid intermediate mapping) B would depend on. (As always, structured states are involved, so independence, inheritance and so on work normally.)

So now we have B(t+1) = function(variables at previous time)

Now, if a single time step of the underlying system is all that's involved, great: we just need to check to see if A is in that list of variables. If it is, then we have causality from A to B.

In other words, if the transition rule is B(t+1) = F(A(t),other(t)) then we have causality from A to B. But if F simplifies to F = f(other(t)) where by the rules of independence other(t) doesn't tell us A(t), then we don't.

If A ==> C, and C ==> B, then A ==> B.

If there is more than one underlying system time step involved (or for a continuous system), then we need to look at a series of time steps (or sequence of moments). Here, starting at the final state, causality can be traced to the previous step. In the end, there must be a chain of local causal relationships which relate the starting formal state A to the final formal state B. I call this a causal chain.

A computational road is also a causal chain, since each step causes the next. At each step along the way, there are "off-ramps" which are counterfactual transitions that are not allowed to occur by the transition rules, and "on-ramps" which are other counterfactual states that would have transitioned to the same state that did occur. The wrong on-ramps (too symmetric with respect to a variable which should be involved in the cause) would spoil the causality. It is OK if an off-ramp (which is not taken) would or would not have led back to the road at a later point by a valid on-ramp.

Back to the example above: Suppose that the aliens _would have_ transmitted back the message "a" instead of "b", but didn't, since I1=I2 in the first place and so they were never contacted.

O2 = TRUE was caused by O1 = a
O1 = a was caused by A = a and B = 0

What if B had been different even though A = a? Then O1 = a would not have happened. But that is OK; since B is an independent variable, we can restrict our mapping to states such that B = 0, which did occur. Within this mapping, O1 = a is caused by A = a.

A = a was caused by Black Box 1's activity
Black Box 1's activity was caused by I1 = I2

So here we have an implementation of a causal chain such that I1 = I2 caused O2 = TRUE.

If I1 had not been equal to I2, it would also have ended up that O2 = TRUE. But at no step along the actual causal chain did we need to explore the path that would have led there.

On the other hand, suppose that the transition rule for O2 had simply been O2(t+1) = f(t). In that case, the sequence of physical states - the activity - could have been the same as in the actual I1 = I2 case, but there would be no causal chain tracing O2 back to I1 and I2.

Formal transition rules of a computation no longer give full information about causal relationships: (I1=I2) ==> O2 = a, together with (I1 not = I2) ==> O2 = a, is equivalent to the simple transition rule O2 = a, and information about causality has been lost.

Causal chains are therefore the more fundamental concept. Like computations, they involve processing of information to cause transitions to specific states. Instead of implementing computations, one should really speak of implementing Structured State Causal Chains (SSCC's), but that doesn't have the same ring to it.