tag:blogger.com,1999:blog-4724592643224262209.post7112780283900864939..comments2023-05-08T02:52:16.953-07:00Comments on Cognition and Evolution: Hedonic Recursion as A Problem of Utility-SeekingMichael Catonhttp://www.blogger.com/profile/01017910055699348111noreply@blogger.comBlogger9125tag:blogger.com,1999:blog-4724592643224262209.post-69951915834086113712013-09-20T09:15:33.932-07:002013-09-20T09:15:33.932-07:00Okay, but hedonic recursion isn't really the s...Okay, but hedonic recursion isn't really the same as being too well-adapted. If anything, from Darwinian perspective, it's maladaptation. A Heroin addict is less functional than a non-addict (if nothing else, he has to pay heroin in addition to all other bills). Someone who plays computer games all day has a harder time holding a job and raising kids. For animals, the literally wireheading rat certainly wasn't too well-adapted!<br /><br />It seems there's a negative correlation between the probability of succumbing to hedonic recursion and fitness. So you could model it as an adaptive weakness, like vulnerability to viruses. If the general ability that led to this weakness - intelligence, foresight, self-modification - brings enough advantages to compensate, extinction doesn't seem to be inevitable.<br /><br />That's not to say AGI can't have weird failure modes that end up with the biosphere destroyed even if its designers didn't want that.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-4724592643224262209.post-77897160284744085392013-09-19T17:35:48.073-07:002013-09-19T17:35:48.073-07:00You don't need perfection, but you also don...You don't need perfection, but you also don't need outside competition. Species have most certainly contributed to their own extinction in the past by being too well-adapted in the near-term. Super-predators and all the organisms that contributed to the oxygen catastrophe (and their own demise) were not wiped out because of aliens. Of course, the biosphere as a whole has kept plugging along so far, it's just certain species that disappeared (and we may be one of those). Of course, those past species were very limited in the changes they could make (they ate everything, or they poisoned themselves by adding a gas to the atmosphere). If AGI is as powerful as some of its proponents promise, the biosphere as a whole is more likely to be threatened, rather than just one or a set of species.Michael Catonhttps://www.blogger.com/profile/01017910055699348111noreply@blogger.comtag:blogger.com,1999:blog-4724592643224262209.post-56226688925822566122013-09-17T13:23:15.793-07:002013-09-17T13:23:15.793-07:00Another possibility is r-selection strategies from...Another possibility is r-selection strategies from high reproduction, combined with Darwinian/economic competition. Spawn a lot of fast-growing kids/mature copies, allow self-modification and then let those die off (painlessly) that can't afford their existence.<br /><br />The problem of staying on a functional path is like the problem of getting a ping-pong ball through a small slit in a large wall. You can try to perfect your aim and never miss, but you can also just fling thousands and thousands of balls randomly.<br /><br />As long as we don't face alien competition, efficiency doesn't much matter. Evolution doesn't need perfection, just survivability.<br /><br />As a bonus, the surplus happiness we want to create comes from the sidelines succumbing to hedonic recursion.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-4724592643224262209.post-48306344333646080002013-09-17T10:19:29.645-07:002013-09-17T10:19:29.645-07:00The latter point is a good one. It seems that it ...The latter point is a good one. It seems that it would still be unpleasant to go from 9-pleasure to 8-pleasure, but not as bad as pain.<br /><br />Pre-commitment devices are useful but in powerfully self-modifying agents, it's difficult to imagine what those could be. People discussing self-modifying AI, which may suffer from this problem, sometime seem to be making directly opposing arguments on this front: "the AIs will be vastly more intelligent than us as we are to worms" and then "and we, the worms, are going to create pre-commitment devices that they can't think their way around".Michael Catonhttps://www.blogger.com/profile/01017910055699348111noreply@blogger.comtag:blogger.com,1999:blog-4724592643224262209.post-36347480914200169852013-09-16T20:55:01.392-07:002013-09-16T20:55:01.392-07:00Redesign could be restricted socially or by pre-co...Redesign could be restricted socially or by pre-commitment devices. A mind could back itself up or create active copy sidelines before self-modifying. Much like people see junkies and steer away from drugs, or ban drugs.<br /><br />Redesign could happen so that pain-driven functions are replaced with pleasure-driven ones without destroying functionality (gradients of bliss instead of pleasure-pain-axis).Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-4724592643224262209.post-26619800686001468362013-08-31T22:31:44.012-07:002013-08-31T22:31:44.012-07:00Thanks for the link - as the implications section ...Thanks for the link - as the implications section of the paper states, "Rather than simply stimulating the reward system in response to traditional goals of food and sex, it would be beneficial to regulate the system and focus it on long-term goals that are more adaptive." Yes it would be beneficial, but when you have complete control over the feedback, programming yourself for the mistaken goals (which are not obviously mistaken) can become disastrous. It's obvious that heroin-like direct pleasure stimulation is bad but if your model of yourself in the world is flawed you can go off the rails as well.Michael Catonhttps://www.blogger.com/profile/01017910055699348111noreply@blogger.comtag:blogger.com,1999:blog-4724592643224262209.post-50953978578896973522013-08-17T04:58:12.983-07:002013-08-17T04:58:12.983-07:00Hi again, I came across an article that reminded m...Hi again, I came across an article that reminded me of this thread on your blog: Case Study of Ecstatic Meditation: fMRI and EEG Evidence of Self-Stimulating a Reward System (http://www.hindawi.com/journals/np/2013/653572/). Just wanted to share that - check out the "Implications" section of the article.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-4724592643224262209.post-23142144557728943082013-07-01T12:48:18.582-07:002013-07-01T12:48:18.582-07:00I think you just hit on the whole problem with thi...I think you just hit on the whole problem with this. That can't be a solution. Value-free and free of desire? Then there's no reason or rationale to modify the system, unless you do so randomly, or eliminate pleasure and pain altogether, in which case, there's no reason (or, a neuroscientist would probably argue, *ability*) to do anything.Michael Catonhttps://www.blogger.com/profile/01017910055699348111noreply@blogger.comtag:blogger.com,1999:blog-4724592643224262209.post-75143462463009450572013-06-19T05:30:32.170-07:002013-06-19T05:30:32.170-07:00Access to self directed goal plasticity (administr...Access to self directed goal plasticity (administrator rights if you wish) might be restricted to a value-free (non-judgemental) approach, free of desire/resistance. This could or could not facilitate evolutionary success, depending on the scale on which this would be applied. If applied widely (society level), this could be an attractor for our social system. If restricted to a minority it might be their demise.Anonymousnoreply@blogger.com