A Possible Solution to Hedonic Recursion in Self-Modifying Agents: Knowledge-Driven Agents

A problem of systems with goals which can self-modify those goals is stability and survival over time. It seems to be a positive that a system could identify goals which are in conflict and modify them to behave consistently in its own benefit, although conflicts can also be solved in favor of the less-survivable goal (example below). The stronger danger is that an agent that can "get under its own hood", so to speak, is able to short circuit the whole process and constantly reward itself for nothing, breaking the feedback loop entirely. This is called hedonic recursion.

An example of conflicting goals: a person wants to be healthy. The same person also really likes eating chocolate. A person with access to his own hardware could resolve the conflict either by modifying himself to make it less fun to eat chocolate, or by to not care about the negatives of being unhealthy. It seems obvious that the first option is the better one for long-term survival, but in the second case, after you modify yourself, you won't care either. And even this second resolution is far less dangerous than outright short-circuiting one's reward center, getting a shot of dopamine for doing nothing. And this short-circuit option would be on the table for a fully self-modifying agent. And, for any self-modifying goal-seeking agent, this will very quickly be realized.

Fortunately or otherwise, this hasn't been a problem for life on Earth yet, because the only way living things here can get rewards is through behavior - because we cannot modify ourselves. The things that cause pleasure and pain are set in stone (or rather, in neurons) and only through behavior (modifying the external environment as opposed to yourself) are rewards obtained. But there are hints in higher vertebrates of small short-circuits - nervous system hacks they have stumbled across which tweak their reward circuits directly. Elephants remember the location of, and seek out fermented fruit (to get happily buzzed). Elephant seals dive rapidly to unnecessary depths to cause narcosis (we think). Primates (including us) masturbate incessantly. And humans specifically have found things like heroin. As we humans learn still more about ourselves and learn how to manipulate the neural substrate, this may be changing. Consequently, if ever humans are able to alter our nervous systems directly and completely, ruin may follow quickly. And indeed, this has been shown with rats: give them the ability to directly stimulate their reward centers with electrical current, and they will do so to the exclusion of all other activities, including those required for survival - hedonic recursion.

In a great discussion at the Machine Intelligence Research Institute website, Luke Muehlhauser talks to Laurent Orseau about how to solve the problem of what kinds of self-modifying agents avoid this problem. The discussion is about how to build an artificial intelligence, but it applies to biological nervous systems that, like us, are increasingly able to self-modify.

One of the theoretical agents Orseau conceived was a knowledge-driven, as opposed to a reinforcement-driven agent, a goal-seeking agent, or a prediction-confirming agent:

...knowledge-seeking has a fundamental distinctive property: On the contrary to rewards, knowledge cannot be faked by manipulating the environment. The agent cannot itself introduce new knowledge in the environment because, well, it already knows what it would introduce, so it's not new knowledge. Rewards, on the contrary, can easily be faked.

I'm not 100% sure, but it seems to me that knowledge seeking may be the only non-trivial utility function that has this non-falsifiability property. In Reinforcement Learning, there is an omnipresent problem called the exploration/exploitation dilemma: The agent must both exploit its knowledge of the environment to gather rewards, and explore its environment to learn if there are better rewards than the ones it already knows about. This implies in general that the agent cannot collect as many rewards as it would like.

But for knowledge seeking, the goal of the agent is to explore, i.e., exploration is exploitation. Therefore the above dilemma collapses to doing only exploration, which is the only meaningful unified solution to this dilemma (the exploitation-only solution leads either to very low rewards or is possible only when the agent already has knowledge of its environment, as in dynamic programming). In more philosophical words, this unifies epistemic rationality and instrumental rationality.

There's a lot more to the argument (you really should read it), but there are several points to be made with respect to this paper.

1) These are not fully self-modifying agents. In this environment their central utility function (reward, knowledge, etc.) remains intact. The solution is to collapse exploitation (reward) into exploration (outward orientation). The knowledge agent can only get buzzed off of novel data, so it has to keep learning. But exploitation and exploration are two conceptually separable entities; so if modification of the central utility function is allowed, eventually the knowledge agents will split exploration and exploitation again, and we're back to reward agents. (At the very least, given arbitrary time, the knowledge agents would create reward agents, to get more data, even if they didn't modify themselves into reward agents.)

2) Orseau's point is taken that if novel data is what's rewarding them, as long as that utility function is intact, they cannot "masturbate" - they have to get stimulation from outside themselves. In another parallel to the real neurology of living things, he states "all agents other than the knowledge agent are not inherently interested in the environment, but only in some inner value." The core of utility is pleasure and pain, which are as much an inner value as it is possible to be. Light is external data, but if you shine a bright light in someone's eyes and it hurts, the pain is not in the light, it's in the experience the light creates through their nervous system. Utility is always an inner value. The trick of the knowledge-based agents is in pinning that inner value to something that cannot arise from inside the system.

3) The knowledge-based agent is maximizing experienced Kolmogorov complexity. That is to say, it wants unexpected information. Interestingly, Orseau says this type of agent is the best candidate for an AI, but such an agent could never evolve by natural selection. He points out that the agents he's using are immortal and don't suffer consequences to their continued operation by any of their experiences. But an agent that can be "damaged" and that is constantly seeking out unexpected environments (ones it doesn't fully understand) would quickly be destroyed. In contrast, Orseau commented that the reinforcement-based agent ends up strongly defending the integrity of its own code. Evolutionarily, any entity that does not defend its own integrity is an entity you won't see very many of (unless the entity is very simple, and/or the substrate is very forgiving of changes. This is why you see a new continuum of viral quasispecies appear after a single year, but why animal species reproductively isolate and you shouldn't hold your breath for, say, hippos to be that much different any time soon.)

4) No doubt real organisms are imperfect amalgamations of all of these agent strategies and more. To that end, Orseau found that the reinforcement (reward)-based agent acts the most like a "survival machine". In his system, I would wager that living things on Earth are reinforcement-based agents with a few goals sprinkled in. (There are many animals, including humans, that startle when they see something snake-like. fMRI studies have even suggested that there are actually specific brain regions in humans corresponding to certain animals - it's really that klugey.) However, of further interest here is that even between humans there are substantial differences in how much utility is to be gained from unexpected novelty, some of them known to be genetically influenced. Some of us are born to be surprise-seeking knowledge agents more than others. The meaning of having multiple genes not at fixation would be useful to investigate. (Only recently valuable in evolutionary time, now that our brains have enough capacity?)


If your goal is to create agents that act to preserve and make more of themselves and remain in contact with the external environment rather than suffering a hedonic recursion implosion, there are a few stop-gaps you might want to put in place.

1. Make self-modification impossible. This is the de facto reality for life on Earth, including us, except for a few hacks like heroin. Life on Earth has at least partly done this, converting early on from RNA to the relatively inert DNA as its code.

2. Build in as strong a future orientation as possible, with the goal being pleasure maximization rather than pain minimization. That way pleasure now (becoming a wirehead) in exchange for no experience of any kind later (pain or pleasure, meaning death) becomes abhorrent. You might complain about the lack of future orientation in humans* but the fact that any organism has any future orientation is testament to its importance.

It could be that we haven't seen alien intelligences because they all become wireheads, and we haven't seen alien singularities expanding toward us because Orseau's E.T. counterparts built their AIs to seek novelty, and the AIs destroy themselves in that way.


Speaking of poor future orientation where reward is concerned: I have seen a man literally dying of heart failure, in part from not complying with his low-sodium diet, eating a cheeseburger and salty, salty fries that he brought with him into the ER.

Churchland's Critique of the Mysterians - Plus Lazy, Middling and Rigorous Mysterian Positions

The mysterian position is often unclear as to exactly the assertion being made, as well as the domains to which it applies. Boiled down, it is this - "The mind cannot understand itself" - although sometimes the claimed unknowability extends further. Often when mysterians throw up their hands, it's in attempts to explain the basis of conscious experience (e.g. Colin McGinn), but often the concept is applied to other domains (language and logic, like Chomsky) or more broadly to all knowledge itself. Patricia Churchland is probably foremost among the mysterians' critics and is having none of this, asking (paraphrasing) "How can mysterians know what we can't know?" (Summary here.) But there are good examples of formally rigorous mysterianism that we already have access to; they're just more limited than what the more aggressive mysterianists are implying.

Churchland regards this surrender to ignorance as no less than anti-Enlightenment, fightin' words if ever there were. While I certainly side with her in terms of approaching the universe with epistemological optimism, the problem is that there are some formal proofs of unknowability, in some domains. She must certainly be aware of these but I'm not aware of any arguments she's made about them. Because it's generally unclear what argument mysterians are making (and their critics are attacking) and what domains they apply to, here is a set of useful distinctions of positions.

Lazy mysterianism. "There is a problem which seems intractable and on which we don't seem to have made much progress; that's because we can't make progress (classically conscious experience)." I think it's clear to all that this kind of giving up is not only un-rigorous, it's pessimistic.

Middling mysterianism, or Human-specific provincial mysterianism, or practical mysterianism. This is the argument that there are things which humans cannot know. But this argument is making a provincial claim about the limitations of the human brain, not about the universe in general. It is without a doubt that the hardware limitations of human brains place constraints on working memory and network size that limit the thoughts we can think, so it's a valid criticism that if we grant that my cat Maximus cannot understand relativity, then there are things that humans cannot understand as well. (Maximus is limited even for a cat but that's not the topic of the post.) The much more controversial part of this brand of mysterianism is when the argument is made not from "commodity" limitations (not enough of something like working memory; you can't think of a two million word sentence all at once) but rather from limitations in the structure of human thought, so that there are logically valid structures that it cannot contain. I'm not going pursue that possiblility, formally or by analogy, but it's worth remembering: an animal with a nervous system designed to mate, avoid predators and find fruit in the African savannah is now insisting that it is a perfect proposition-evaluating machine that can understand everything there is to be understood in the universe. (At the other extreme are people like Nagel, who say that the lowly origin of our brains means we can't know anything.) Especially if other kinds of nervous systems on the planet cannot understand everything, it seems the burden is strongly on those who would make the argument that everything is suddenly within the reach of humans. More simply: if you don't think you could teach the alphabet to Maximus the stupid cat, the burden of proof is on you to explain why everything can be taught to a human. What is so fundamentally different about the two?

From Gary Larson's Far Side.
The key question here is whether it's possible to tell random noise in the universe from something that is comprehensible but beyond us; for example, an advanced self-programming computer or a superintelligent alien, trying to explain some theory to us. You might even say that something is unknowable with current knowledge, because that causes a change in brain state - think how impossible it would be for the otherwise-bright Aristotle to understand the ultraviolet catastrophe without the intervening math and physics - but obviously this didn't mean it couldn't be understood, ever, period. But now we're back to lazy mysterianism, because eventually the UV catastrophe could be understood, so this is useful - the difference between lazy and provincial materialism is whether you can modify your brain state with experience in a way to make you understand it, as Planck and people after him have.

Some would object here, and pursue an angle that argues "if a superintelligent alien tells us something that's 'beyond us"', but we can differentiate nonsense against things that we just don't understand, then we actually do understand it" - which brings us to our next point.

Rigorous mysterianism. Rigorous mysterianism is true. It seems we have a very good reason to think that there are some instances where we can't in principle know things. Turing's halting problem is about as good an example of this as any. Here we have a well-studied formal proof that we cannot know whether an input will ever produce an output. Formal mysterianism! It certainly seems that we have a good reason to believe this is an insoluble problem, because we have a rigorous demonstration of it. An over-optimist might say "Actually the halting problem is just lazy mysterianism. Right now in 2013 it seems like we can't solve the halting problem but that's just a limitation of modern knowledge." Let's be consistent then; by such an argument, all things we don't know will eventually be knowable, including how to go faster than light, how to know both the position and momentum of a particle, and any other number of things. Attacking mysterianism cannot mean rejecting even all formal positive limitations on knowledge, or sequitur quodlibet, anything follows.


Commodity Limitations on Human Cognition and What They Mean For "Understanding"

The interesting points are to be found in the middling, provincial position, because this forces us to define what it means to know or understand something. I may be conflating two terms here: is conscious understanding required for knowledge? Is understanding even a real thing? Think of all the times you've thought you've understood something, really had a genuine honest sense of it, only to have reality eventually show you otherwise. While this doesn't mean that understanding is just a sense experience and always a delusional one at that - though that's one possibility - thinking you understand something is not a solid guide to whether you are correct. Only external reality is, hence the empirical scientific method. This difference, between understanding something and knowledge of something (or the possibility of knowledge about it), may be key to clarifying mysterian positions. And this approach to clarifying the cognitive processes involved in knowledge, and the physical structures that underlie them, should appeal to neurophilosophers like Churchland and help us clarify what we're talking about.

Here's a thought experiment illustrating a commodity limitation to human cognition. An alien lands outside your door and demonstrates some physically amazing thing (teleportation), and says, "Now I'm going to explain it to you and you will understand it. The shortest way to explain it to a human is a twelve billion word sentence." (Which takes roughly 200 years to listen to if you sleep 8 hours and day and listen to the alien for the rest.) That means you will never understand teleportation, period. That may be different from humans not being able to understand it in general.

So you say, "I'll be dead by then, and you might have a nice voice for an alien but listening to you sixteen hours a day the rest of my life would be unpleasant anyway. Can I get a large group of people to help me?" And the alien is nice so it says "Sure." So you start. Because some are on night shift you can do it in 130 years. So in 2143 they get to the end of the sentence, build a machine, and bang-o, teleportation.

In this case: does anyone still "know" how the teleporter works? Obviously yes! you say, they built it! But everyone heard their own three-month subordinate clause, has their own piece of the machine, and it plugs into the rest; no one has all the knowledge. They know that when the parts are put together in such and such way and you put Maximus the stupid cat in this box and press this button, Maximus the stupid cat comes out on the other end - but Maximus is even lazier than he is stupid, so soon enough even he figures out that when you press the red button, you can go to the other room where the food dish is without having to walk all the way there. Unrealistic? Some dogs are smart enough to take the subway in Moscow. They just haven't built their own yet, and no school has been able to teach them. Similarly, it's not a teleporter, but this cat has certainly figured out how to amuse itself with a toilet. Do they understand the subway and the toilet? Do you understand how your phone works? Does any one person at a smartphone manufacturer know how their product works?

Let's go back to the initial alien landing. Now let's say the alien isn't such a chatty Cathy, and the sentence only takes forty years to say, but now the alien is more fixated on having the first person they meet, and just that person, as an audience. For the sake of advancing human technology, he agrees to become a transcription-monk, giving his life to writing down the alien's long sentence. At the end, voila, the monk builds a teleportation device. Now does he know how it works? Does he understand it? "Now this time it's a home run," you say. "He clearly understood the whole sentence and he built the thing all on his own!" Let's come back to this in a bit.

Of course you see the trick here. The long alien sentence is the history of science and engineering, standing on the shoulders of giants. The trick that humans have and that animals don't, or at the very least they don't have it as well-developed as we do, is language, which allows us to overcome those commodity limitations on knowledge and even our own mortality, in order to cooperate with others and advance our ability to predict the behavior of the universe and choose actions accordingly.

The best analogy to the alien sentences are mathematical proofs. Andrew Wiles solved Fermat's last theorem; the proof is over a hundred pages. The vast majority of humans do not understand it (and I submit, could never have understood it, even with the same schooling as Wiles.) But does Wiles understand it? Don't roll your eyes! I'm not asking whether he bumbled about scribbling randomly until he said "Oh dear, I finally seem to have solved it, but don't ask me how". But Wiles, brilliant though he is, still had to write it down (of course) because he understands - that is, holds in working memory - the steps one or a few at a time, not the whole proof. His network density is what differentiates him from subway dogs and from me. Similarly, the people who make smartphones don't understand everything about solid state physics or even their own product, but individuals understand the pieces (because of their networks) and how to plug them together with other pieces they don't understand (like dogs on the subway).

The knowledge of how to make a smartphone clearly exists in the world, yet no one understands it (not all of it; to claim otherwise is to claim that I do actually understand Wiles's solution because I can understand one page of the proof, and believe mathematicians when they tell me it fits with the rest). There's too much information, and even Wiles can't grasp nearly all of it simultaneously. For that matter, even multiplication of large numbers cannot be understood in one piece, but we can still tell it's correct. Understanding as it's usually described, and knowledge - knowledge verified by behavior, by experiment and how to do something - are two different things. There are two arguments buried there that I'll unpack.


CONCLUSION

First, whatever we mean by knowledge it is possible for humans to have, quantitative commodity limitations cannot rule out such possible knowledge. That is to say, your limited working memory (which language helps us overcome) does not determine what is outside of possible human knowledge. Time to understanding doesn't count either; even if we can't understand it now, people might understand it later, building on what we've already learned. To cling to "understanding" as meaning we can think all the thoughts together, we must argue either that there are people who have in their heads, at one point in time, the entire workings of a smartphone (which there are not), OR we must consider smartphones as mysterious, which is stupid, because we make the things. Whatever provincial (human-specific) limitations exist on possible knowledge (if any), they must be based on network density and quality. This is what allows profoundly improved language in humans, and why cats can't do multiplication, and why almost no humans can understand the solution to Fermat's last theorem, but a few can.

Second, and more controversially, it may be more useful to define what we mean by knowledge as information that affects decisions and behavior that consistently produce an expected result, regardless of the subjective experience of this information, i.e. the sense of understanding. Brains and computers both use representations that allow them to behave in ways that interact with the rest of the universe in a predictable way; this is knowledge, even if it's incomplete or perfect. So Wiles, Moscow dogs, and I all have knowledge of how to ride the subway. Wiles and I have knowledge of smart phones. Only Wiles has knowledge of how to solve Fermat's last theorem. Even Wiles does not understand it in full, only in tiny pieces. It seems a short jump to say that it would not be difficult to build a machine that can navigate the first two problems; and we already have systems that can check the last one (but not solve it in the first place). That is to say, computers have representations that let them solve those problems, and therefore have knowledge. But computers don't understand things (have a subjective experience of logical perception), whereas humans can have this sense, but it's notoriously unreliable, and certainly not grounds for claiming to others - and limited to small pieces of most trains of reasoning.

Why Do We Care About Consciousness?

The problem of understanding consciousness is traditionally broken into the easy problem (how the brain works as a computer) and the hard problem (how the brain creates subjective experience). Indeed, the hard problem seems so intractable, and progress toward a solution seems so tricky to measure, and because it's not even clear what kind of an answer will explain it (we really don't know how to start getting there from here) that it's been called more of a mystery than a problem.

Here are possible reasons the hard problems still seems more like a mystery than a problem:

- It's an incredibly difficult problem: our science so far is not nearly sufficient to explain it, and/or our brains have difficulty with these explanations. (Whether this is a feature of brains in general or humans brains right now is another question.

- There are bad ideas obstructing our understanding. This is an active failure of explanation, rather than a passive failure as above. We have a folk theory or theories (a la Churchland) about subjective experience that we don't know we have and/or are not ready to discard), which is complicating our explanations. Our account is like trying to explain chemistry with vitalist doctrine, or the solar system with the Earth at the center (probably worse than that last one, which can be done, it's just pointlessly difficult and messy.)

- The first-person-ness of experience is a red herring. This may be one specific bad idea as above. When you're explaining an object in free-fall, no one worries that you yourself are not experiencing acceleration. The explanation works regardless of where the explainer is.

- Some non-materialist irrational truths hold in the universe. If that's "true" I don't know how we could ever know it.

Explaining things necessarily involves trying to build a bridge from what we already know to the not-yet-understood thing; but so far this endeavor has the flavor of checking internal consistency between what we already know about nervous systems. It seems that we're mostly motivated to explain consciousness, because we're bothered by the resistance of this idea to explanation. (I certainly am.) But if we don't know where we're going yet this kind of approach might not get us very far. One obscure but interesting example of obsession with internal consistency of theories comes from pre-Common Era China, where Taoist logicians agonized over the relationships between the properties describes by yin and by yang. Yang things are both hard, and white. So what, then, is the logical relationship between hard and white? They're both yang, they reasoned, so there must be such a relationship.

Of course we wonder, where did the Taoists think they were going to get with that kind of tail-spinning if they thought they were trying to answer pretty deep questions? And so we should turn the same question on ourself: why do we care about consciousness? What impact will the theory have once we understand it?

The obvious answer is that it's ultimately a moral question. While it's not clear whether affective components of experience (at base, pleasure and pain) are necessarily a part of consciousness, they certainly are possible in conscious beings, because I experience them, and so even though saying this makes verificationalists mad, I give credit to other apparent first-person viewpoints of other living things (e.g. other humans, dogs, cats) that they experience them too.

Consequently, if neuroscientists of the future that build nervous systems, and AI engineers (if they're two different professions) believe that their lab subjects can experience consciousness, then it becomes incumbent on them to understand what they're experiencing. If we're capable of creating things are conscious, we have to avoid creating ones that are predisposed to suffer. Indeed with such an explanation we may take notice of other structures in the universe that can suffer but that we didn't even realize were conscious before.

Once we understand the material basis of conscious awareness (if there is such an explanation), then we can start asking some heavily Singulatarian-type questions - whether mind uploading, the transporter problem, etc. are really extensions of a self, or just copying, and whether there are meaningful differences in those alternatives.

Finally, understanding the basis of consciousness may allow us to alter the structure of conscious objects in a way that decreases their suffering and expands their happiness - from first principles. Currently we're limited to what I hope will seem in the future to be very limited, clumsy manipulations of nervous systems to decrease suffering - e.g. taking consciousness away with anesthetics, NSAIDs, anti-psychotic medicines, talk therapies, and beyond medicine, all the behaviors that we engage in minute-to-minute to enhance fluorishing and decrease suffering in ourselves and the beings around us.

Hedonic Recursion as A Problem of Utility-Seeking

Summary:  We try to align our goals so they are consistent with each other but still often have contradictory ones, often owing to hard-wired aspects of biology.  Therefore, we continually try to adjust our preferences to be consistent (i.e. rational).  The ultimate end of rational goal-directed behavior then would be an ability to absolutely "edit" even our most profound preferences.  This may result in a system not conducive to continued survival; that is, if the source of our utility can be adjusted to itself increase utility, then our decisions lose all input from the world outside our nervous systems.  For this reason self-awareness may universally be an evolutionary dead-end.


A problem of agents with goals is that goals can be contradictory.  Goals are behavioral endpoints that maximize pleasure (utility) and minimize pain.  An evolved system like a human being should be expected to have many conflicting goals, and indeed we do, especially in areas where the goal's context has changed rapidly due to culture (e.g., "always consume high quality forage" vs. "stay slender to maintain health").  Of course this is only one of many examples.

Unfortunately, many goals are fairly hard-wired; no one has ever decided to be drawn to sugar and fat, and it's only with great difficulty and cleverness that a few of us in the developed world find hacks to prevent one of the competing goals from erasing gains from the others.  Furthermore it should be emphasized that we can't decide NOT to be drawn to sugar and fat.  We don't have the ability to modify ourselves to that degree; not yet, anyway.  (A hypothetical technology doing just this is discussed in more detail at Noahpinion.)

Would such an ability be a positive one?  It's anything but clear.  In this situation, we could make sugar and fat less appealing (or activity less painful), and/or make hunger pleasurable.  Less obvious but equally valid, you could program yourself to just stop worrying about heart disease and diabetes and gorge away.  To get at the core issue, what we're trying to do is maximize our utility by editing our goals - but there you see the circularity.  Achieving goals are what increase utility!  In some cases like food consumption, it's a relatively easy fix.  But let's consider mortality.  Assume that at some future time, you have the ability to edit our goals (our utility-providing endpoints) even at the most basic biological level.  However, also assume that there are still terminal diseases and finite lifespans; you have contracted one of these and are approaching the end of your own.  Unpleasant?  Of course not!  Make death something you look forward to!  Make your breakthrough cancer pain like an orgasm!  Why not?  Hurry up and let me suffer and die already why don't you - ahhhh, that's the stuff.  If you object "pleasure should only be a feedback mechanism and has to reflect reality", the completely-editable hedonist will bluntly ask you why, and probably take pity on you for not being able to get into your own source code.

In point of fact, why not make the most minute actions throughout life (or even absence thereof) result in untold pleasure?  And here is the problem.  Once we can manipulate our utility at such a profound level, we may be in dangerous territory.  Most of us can't control ourselves around potato chips.  It is likely that being given the key to our own reward centers this way would result not just in a realignment of utility with respect to our changed environment, but in giving ourselves easy utility-points for anything - and then what happens to motivation?  Why do anything, except give yourself more utility points?  (This is called hedonic recursion., and this is why it may be a good thing that for the most part, what we like and don't like is biologically tied down and non-negotiable.)  Even systems to assess the rationality of preferences (like the von Neumann-Morgenstern axioms) are only about internal consistency.  An individual that "solved" his own preferences in such a way that was consistent with death being desirable is not irrational.  No, he won't leave many offspring, but that wasn't what gave them utility after he got under his own hood and did extensive rework.

The ideal "logical" hedonic solution would seem to be giving ourselves pleasure in such a way that our likelihood to survive over time is increased, to increase total pleasure exposure.  (This satisfies our moral instinct that it's better to make yourself stop liking junk food than to stop worrying about dying.)  At first glance, one might argue that this is what evolution has done anyway, notwithstanding any mismatch effects from recent rapid cultural changes to our physical environment.  But evolution optimizes survival of genes over time, regardless of what conscious individual(s) they happen to be in at the moment, and pain and pleasure are always subservient to that.  Pain and fear just as often (perhaps at least as often) motivate survival behavior.  Here we're explicitly minimizing one in favor of the other.  To illustrate the difference in the two processes (and the recursion inherent in one):  evolution uses pleasure and pain to maximize gene numbers over time.  Pleasure uses pleasure to maximize pleasure.  One of these looks very much like a broken feedback loop.

Very speculatively, assuming that such goal plasticity is possible (perhaps after mind-uploading) it could be that hedonic recursion is not something unique to the way our nervous systems are put together, but rather a fundamental problem of any representational tissue that develops a binary feedback system.  So far, we've  been protected by the highly manufacturing-process-constrained substrate of our minds.  If so, over time such systems will increasingly learn how to please themselves and shut of pain, independent of what's going on in the outside world, to a point where they're so cut off from stimuli that causes them to survive less that they disappear.  Self-awareness always goes off the rails.  This is one possible solution to the Fermi paradox:  that pleasure inevitably becomes an end in itself, divorced from survival, and the experiencing agents always disappear.

Memory Traces Observed in Vitro

It's less than implanting and reading a memory in vitro, but researchers were able to tell most of the time based on output which dentate neurons were being directy stimuated. Probably not now, but at some point, we're going to have to start asking hard questions about what structures necessarily produce consciousness (whether in vivo, vitro, or computationally), because then this work takes on a moral dimension.

Subjective Experience and Memory

The strange properties of subjective experience are nowhere more evident than in the use of drugs which prevent long-term potentiation - or destroy the capacity for consciousness in the first place. Is there anything it is like to be under anesthesia, or do you just forget that you were tortured?

Note, Harris's focus here is about free will and morality - and to his detriment he completely neglects the robustness of epiphenomenal accounts to the neuroscience experiments he's citing, probably because few people would find an epiphenomenalist account of free will no less distasteful than a total absence of free will. But the question here sends him down an interesting road, and to my knowledge the first clinical test of the global workspace hypothesis of consciousness was in fact by a team of anesthesiologists.

In the Land of the Blind

...the one-eyed man must remember the majority is always sane. The Country of the Blind by H.G. Wells explores how an entirely blind civilization might view the universe, and how sighted people might interact with them. The Churchlands' infra-people, while argumentative, seem quite diplomatic by comparison.

Ultrasound, Neuronal Excitability, and Hallucinations

A few years ago it was announced that a U.S. patent had been filed by Thomas Dawson on behalf of Sony for direct neural input of sensory information. There are a few: the most recent related patent by the same inventor here, but the one that I believe attracted media attention is here).

The patents describe a method of stimulating multi-modality sensory experience through the use of sound energy. The basic idea is that neuronal excitability can be increased with ultrasound (review here) although to date it seems that all the work has been either with CNS neurons in culture, or PNS neurons, rather than CNS neurons inside a spine or skull. Obviously if these patents represent functioning technology the implications are profound. For that reason I scrutinized 6,536,440 for evidence that the concept had in fact been reduced to practice, which (naively I'm told) I had thought was still a requirement for the issue of a patent. There's precious little in the document to suggest anyone is ready to build a transducer capable of producing any sensory experience in subjects, let alone a coherent one.

Of course ultrasound is already used in medical imaging all the time, in a similar range to that reported in the review (optimal transcranial transmission of ultrasound at 7 x 10^5 Hz, but in vitro studies showed neuronal excitability changes at higher frequencies around 2-7 x 10^6 Hz. Medical imaging ultrasounds use frequencies up to 10^7 Hz, but the intensity range is 1-10 W/cm^2, and the imaging device is rarely applied to the skull (useless for imaging, because bone blocks commercial ultrasound). That said, I'm unaware of anecdotal reports of patients hallucinating by any modality during ultrasound imaging, a very common outpatient procedure, and quick search of Pubmed reveals no such cases among the first 30 articles.