Brain Power

Supercomputers

In 1984 when big brother was meant to invade our privacy I was a graduate student in Glasgow working on lattice gauge theories. As part of the research towards my doctorate I spent a week on a special mission to Germany where I was allowed into a secret nuclear base to borrow some computer time on a Cray-XMP. It was the world’s fastest supercomputer of the time and there was only one in Europe so I was very privileged to get some time on it even if it was only a few hours of CPU. Such resources would have been hugely expensive if we had to pay for them. I remember how the German’s jokingly priced the unseen cost in BMWs. The power of that computer was 400 Megaflops and it had a massive 512 Megabyte ram disk.

The problem I was working on was to look for chiral symmetry breaking in QCD at high temperatures and densities using lattice simulations. In the last few years this has been seen experimentally at the LHC and other heavy ion accelerators but back then it was just theory. To do this I had to look at the linear discretised Dirac equation for quarks on a background of lattice gauge fields. This gave a big hermitian NxN matrix where N is the number of lattice sites times 3 for the QCD colours. On a small lattice of 164 sites (working in 4D spacetime) this gave matrices of  196608 square and I had to find its smallest eignevalues. The density of this spectrum says whether or not chiral symmetry is broken. Those are pretty big matrices to calculate the eigenvalues of, but they are sparse matrices with only 12 complex non-zero components in each row or column. My collaborators and I had some good tricks for solving the problem. Our papers are still collecting a trickle of citations.

tianhe-2

Thirty years later big brother has finally succeeded in monitoring what everyone is doing in the privacy of their own homes and my desktop computer has perhaps 100 times the speed and 30 times the memory of the Cray-XMP, which makes me wonder what I should be doing with it. The title for the fastest supercomputer has recently been taken by China’s Tianhe-2 which has been benchmarked at 33.86 Petaflops and it has a theoretical peek performance of 53.9 Petaflops so it is about 100,000,000 times faster than the Cray. This beats Moore’s law by a factor of 5000 which may be in part due to governments being willing to spend much more money on them. The US who more commonly hold the record wont be beaten for long because the NSA is said to have a secret and very expensive project to build a supercomputer to surpass the Exaflop mark in the next few years. I doubt that any HEP grad students will have  a chance to use it.

This begs the question: Why do they need such powerful computers? In the past they may have been used to simulate nuclear explosions or design stealth fighters. Now they may be needed to decrypt and search all our e-mails for signs of dissenting tendencies, or perhaps there is an even more sinister purpose.

Artificial Intelligence

When computer pioneers such as Von Neumann and Turing conceived the possibility of building electronic computers they thought it would be easy to make computers think like humans even though they had no idea how fast computers would become. This turned out to be much harder than expected. Despite some triumphs such as “superhuman” chess programs which can now crush the best grandmasters (see discussion at World Science Festival) the problem of making computers think like us has seen little progress. One possibility that looked promising back in the 1980s was neural networks. When I left academia some of my colleagues at Edinburgh were switching to neural networks because the theory and the computing problems were very similar to lattice calculations. Today their work has applications in areas such as facial recognition but it has failed to deliver any real AI.

Now a new idea is raising hopes based on the increasing power of computers and scanning technologies. Can we simply map the brain and simulate it on a computer? To get a flavour of what is involved you can watch this TED talk by neuroscientist Sebastian Seung. His aim is to simulate a small part of a mouse brain, which seems quite unambitious but actually it is a huge challenge. If they can get that working then it may be simply a case of scaling up to simulate a complete human brain. If you want to see a project that anyone can join try OpenWorm which aims to simulate the 7000 neuro-connections of a nemotode worm, the simplest functioning brain in nature (apart from [insert your favourite victim here]).

Brain Scans

An important step will be to scan every relevant detail of the brain which consists of  100 billion neurons connected by a quadrillion synapses. Incredibly the first step towards this has already been taken. As part of the European Human Brain Project funded with a billion Euros scientists have taken the brain of a 65 year old women who died with a healthy brain, and they have sliced into 7404 sections each just 20 microns thick (Obama’s Brain Mapping Project which has had a lot of publicity is just a modest scaled down version of the European one). This is nearly good enough detail to get a complete map of the synaptic connections of every neuron in the brain, but that is not clear yet. If it is not quite good enough yet it is at least clear that with an order of magnitude more detail it will be, so it is now only a matter of time before that goal is achieved.

If we can map the brain in such precise detail will we be able to simulate its function? The basic connectivity graph of the neurons forms a sparse matrix much like the ones I used to study chiral symmetry breaking but with about a trillion times as many numbers. An Exaflop supercomputer is about a trillion times more powerful than the one I used back in 1984, so we are nearly there (assuming linear scaling). The repeated firing of neurons in the brain is ( to a first approximation ) just like multiplying the signal repeatedly by the connection matrix. Stable signals will be represented by eigenvectors of the matrix so it is plausible that our memories are just the eigenvalue spectrum of the synaptic map and the numerical methods we used in lattice gauge theories will be applicable here to.

However, the processes of logical reasoning are more than just recalling memories and will surely depend on non-linear effects in the brain just as the real physics of lattice QCD depends on the highly non-linear interactions of the gauge field. Will they be able to simulate those for a human brain on a computer? I have no idea, but the implications of being able to do so are enormous. People are starting to talk seriously about the moral implications as well as what it may bring in capability. I can understand that some agencies may want any such simulations to be conducted under a veil of secrecy if possible. Is this what is driving governments to push supercomputer power so far?

It would be ironic if the first true artificial intelligence is actually a faithful simulation of a human brain. No doubt billionaires will want to fund the copying of their own brains to giant supercomputers at the end of their lives if this becomes possible. But once we have the capability to simulate a brain we will also start to understand how it works, and then we will be able to build intelligent computers whose power of thought goes far beyond our own. Soon it may no longer be a question of if this is possible, just when.


27 Responses to Brain Power

  1. Vladimir Kalitvianski says:

    Computers can be faster than we are and be perfect in logic, but they will never be “believers”. Beliefs brake logic and impose solutions (prejudice). By the way, have you seen this lecture by Feynman? He speaks of “thinking business” at t=1:07:10 s.

    • I’ve noticed a strange belief in the scientific community – scientists seem to claim they don’t “believe” in anything. I imagine this claim is a refute of religion, mysticism and New Agers. After all, scientists DO believe in atoms, the Earth orbiting the Sun, etc. etc. They should also be extremely careful when they criticise every aspect of religion and mysticism. For example, think of the strange things that could result from the universe being a unified field in which all seemingly separate objects in time and space are actually one thing. Centuries from now, this could render today’s scientific reluctance to believe a quaint symbol of a primitive culture … and expose today’s science as a dinosaur unable to adjust to the modern world of those future centuries. Tomorrow’s world could regard today’s scientists in the same way that the science of 2013 thinks of priests belonging to the Middle Ages.

    • Tienzen (Jeh-Tween) Gong says:

      Vladimir Kalitvianski: “Computers can be faster than we are and be perfect in logic, but they will never be ‘believers’.”

      This is not true. You have believed in something which is not based on any true knowledge (both theoretically and evidentially). I have worked on AI (artificial intelligence) and the article “The Linguistics Space (II) — the Intelligence (http://www.prebabel.info/aintel.htm )” discussed some of those theoretical issues.

  2. Vladimir Kalitvianski says:

    R. Feynman on computers: http://www.youtube.com/watch?v=EKWGGDXe5MA

  3. I find it interesting that in the early 90′s the Cray computer was said to have the brain power of a mouse.

  4. Very interesting disclosure !

  5. Joel Rice says:

    speaking of brains, NPR had a program on ‘hearing voices’ with a guy who could follow 4 symphony orchestras playing all at once !, and distinguish them spatially. A conductor could follow one, but got a brain cramp with two – as does just about everybody else. So, i am not holding my breath.

  6. Wes Hansen says:

    Yeah, I’m a big fan of Neuromancer and can’t wait to see AI hit the mainstream but I would agree that we seem to have a ways to go . . . (http://neurosciencenews.com/neuroanatomy-sensory-information-thalamus-269/)

    • Orwin O'Dowd says:

      That’s an interesting report – Luria mentions the tissue layers and I always wondered what they were up to. So the “hard-wired” connectionist program got completely trapped in there, and in fact we haven’t the faintest “hard-wired” idea how any synthesis of sensory modalities occurs, or anything therefore in the range of classical association or stimulus-response continuity.

      Then its back to tradition, where the “deep” impact is (Arabic) wa’hm, popularly a whammy, the gut-level shock that primes fight or flight. Also called the Moral Sense, the real touchstone for David Hume, not to mention Leibniz, Wolff, Lambert, Kant, Mill….

      The tricky part is, that’s all Intensional logic, not the Extensional stuff of the Ikky Professional Consensus. But as Lambert insisted, only Intensional logic can handle possibilities that may or may not be realized, so the Ikky Paranoid Paradigm has zero application in technological development, and merely drives pure science over the edges of realizability (or off to Mars).

      • Orwin O'Dowd says:

        Correction, that’s Arabic wahm. Like Phil, I have a slight disability which scrambles syllables. But now I see that the sensory signal splits into an advanced and a retarded wave, a meta-phasis or after-phase, which is the name for my particular problem. Seems it doubles back through memory and may carry traumatic echoes, as in PTSD. Anycase, that’s a delayed feed-forward, and then supervenes on the advanced wave, all within the scope of one sensory quale!

        I am very glad to understand all this, and find an emphatic echo in Seneca, whose philosophical path lead both into the self and away from the self. He wrote in Latin, and wrote tragedies, like Satre, something of an existentialist, and now speaks to us past Satre and Merleau-Ponty. I mean he knew that you can have relevant knowledge which still does nothing for you, while Freud tripped over that late in his life in a huge admission of defeat.

        So there is potential around here for a whole new kind of structuralism, but its not exactly a happening thing yet. Actually Meta-Physics was just a Latin name for the text placed After the Physics (of Aristotle), which is much where we are after all these years!

  7. Joel there seems to be a number of things a human can do at once such as how many numbers memorized… say so many as if a pattern, a low number. But there is a case where a radioman could send and receive 9 incoming and 9 outgoing Morse code messages at the same time. Is this human intelligence any more than the rain man in recognizing say 60 base patterns more efficiently- that perhaps a better intelligence or just a more general filing system as Feynman discusses in the video link Vladimir posted above on heuristics.

    I was disappointed in that Feynman lecture save maybe for historical reasons- it makes me glad I did not get caught up in that last century of what is technology and physics- but the cost of this is that I find it hard to suspend belief that would make science fiction more enjoyable. I expected Feynman would address it from a quantum viewpoint, not a matrix of a filing system of choice between a red and blue binary pill. What is quantum theory but the bra and ket as a way to generate functions. Still what is between it that some desire to keep an almost religious or cultural mystery is the c in bra(c)ket.

    Now, Phil your brain power article and your own testimony of experience in using computers to solve a problem inside the the game of this programming thus apply it to say particle physics raises deep and parallel questions that are current to our day such as what is the use of all the data if big brother does not know how to use it or what can be kept secret with a purpose.

    Lately the frontier is to map the brain but would that simulation turn our as such an artificial intelligence you ask? Consider this from my view I did not know you could use the computers the way you described the way to view such filing systems and methods hauntingly similar as information processing goes than my own which is to say the patterns we need to find are a little more universal than from his view imagines. His Tee shirt says a lot about structure but so much is lost as is gained in the translation to simplicity so we may fear ourselves in fearing the machines.

    Now the values you sought for with your computer time come up with matrices in matrices which in the cloud view of quantum theory certainly suggests all the non-linearity as fuzzyness to sort thru- but the numbers you find say as QCD are more general in pattern than simple reduced binary systems (regardless of imaginary number modeling which as you stated finds “chiral symmetry breaking”. So what if we then see the pattern of your results finding ten fold symmetry as in groups of say 5 dimensions… The brain model to simulate would at least include the DNA and as we know it too is multilevel in the reading and not a simple genome nor isolated from its environment. Where is the memory? Why do we need time in a static natural filing system? Could we replicate the brain without it being in fact indistinguishable from a flesh and blood brain?

    Lately we can transmit ten fold channels in fiber optics if we spin the light in them in transit… the symmetry of the double helix comes to mind. In a sense this level of complexity where biology meets the cosmic code does seem to say machines think and the universe could have a moral code within as dumb or insane matter on the outside.

    The next generation of physic will be quite an amazing place, as Feynman said a knife can be used for good or ill. I like what Kierkegaard said: To the corkscrew the knife is crooked… the ultimate source is deeper than just the substance gained by moving files around as if it will emerge from our inadequate ideas of what is economics and democracy.

    • hansvanleunen says:

      Computer hardware does not pose yet undissolvable problems, but software does!
      Most hardware systems are designed and built in a modular way from components that easily couple and are readily availble on a components market that ensures a high quality/price ratio. This is NOT the case for software. Software components that can easily be coupled and that can join in more complicated components do not exist.
      If these software components were available, then software design would be orders of magnitude more cheaper, more reliable and faster.

      • Stephen Crowley says:

        hans, this is why microsoft and their ilk are evil and the work the free software foundation @ fsf.org is not

      • Hans, Indeed!

        I mean what is zero and one but a way we see a little positive or negative voltage in the hardware (so in the software we can try to organize the filing system as Feynman pointed out treating this new world as the knowable on this side of the classical)?

        So we seem to have two kinds of physics laws and without a unified physics or deciding if it is possible or is a matter of our placing meaningless patterns on the world or seeing them in our perceptions (where is the difference at what point where we still debate the artificial from the natural?) or choosing but one of them which one is relevant or primary in nature’s laws?

        Let us face it, F=ma is an ideal or approximation and we as in Newton’s gravity may have to consider modifying it with some sort of more general relativity to begin to understand from our position in the reference frame just what we are seeing.

        But Ohm’s law is a different story. I mean within a certain stated tolerance we can pretty well feel assured we can put a component in circuit and trust it as an absolute value with predictable results (so my father sends me as a kid to pick out resistors for his projects- I mean I did not change its properties by reading the color code he taught me as I observed it).

        It has been a long time since I could repair televisions by some simple part let alone open a chip so easy now to replace for pennies. But who knows, Pauling found it humorous electric things would break down in his presence and I knew someone that just did not get along with electrical machinery. How do our brains really learn to move our artificial phantom limbs anyway?

        Are you sharing your insights or there is some point you are making that goes over my head? If so my apologies Maybe that there is a third kind of law where Macrohard covert corporation lets us take things fore granted with good intentions dumbing us down for the sake of the mass ease of physical and user ergonomics . Lord knows hacking the laws of the universe does introduce a little uncertainty as all us lesser gods play dice and add to the watching the pot boil. (of which liquid helium will not boil as the old wives knew when it is watched).

        Third law: the more I lean an operating system and catch up with its growing complexity – the dumber I get. (some say this includes what happens after 4 years of formal education as many young professor assistants seeking a place, job, or tenure have said to me they envy the time I had for reading)

        So in the world where we find mysterious touch or have to cross and focus our eyes to decide what is visual illusion or find more of ourselves to compensate for the cost of retrieving memories that we can find something new to us again as it causes in us an equal and opposite human reaction, we tend to stay in the same old habits as bodies in lazy rest or faster than light motion unaware it helps to pretend we have infinite lives in the games like Mario and have to go back to the old QWERTY that most of us can read or this time around going back to the drawing board finally realize we need to include the start button in the space needed to navigate the near and distant cloudy touchscreen only as smart as our smartphones.

        As far as I know while someone can see me typing this it is my fingers originating this message :-)

  8. kneemo says:

    Philip, have you looked at D-wave’s developer pages? I think their current quantum computational system is at 503 qubits, but one can still get a sense of their utility in the development of A.I. applications. D-Wave Dev-Portal

    • Philip Gibbs says:

      Can they do 2^503 computations simultaneously and factor 1006-bit numbers instantly using the Shor algorithm? I guess not but then what can they do?

      • Stephen Crowley says:

        generate hype and cause a bunch of CIA/NSA worshippers to spooge in their pants

      • Orwin O'Dowd says:

        No the strongest computer remains the mechanical Babbage engine which grinds out math tables (!) at 32 DECIMAL places. And the Stepping rating for an x86 chip comes straight of the Heisenberg matrix, so its all strictly quantum already and teh D-wave is not provably faster. Edgar’s right again, optics is the next horizon.

  9. Marni says:

    Oh, I also have some papers in lattice QCD, but I don’t think that helps with studying emergent geometry, for instance, since one takes the usual gauge symmetries from the start. Sparse matrices are pretty cool, though.

    • Philip Gibbs says:

      My LGT background helped me think about randon graph models and then matrix models.

      Hope you have seen Nima’s talk at srings2013. Polytopes are back.

      • Polytopes were always with me since I first read Felix Klein who did the finite part of what Lie did…in 68 Maybe that is why what I am saying rings a bell…. I think one of the first posts I made on your blog was that Marni was on the right track… now, what is a sparse matrix? More homework… thanks man.

        Oh, when we map 2 to the 512 or so from my view that results in 16 what we think of as emergent dimensions. I suppose in the old days students would actually map the matrices that would fill up a basketball court… but Marni was right in that we have to figure things out on the current level first before we go to possible higher things.

        In any case, their is a deep question as to if in the end it is just patterns we impose or discover where none may be there or if in the workings of nature some patterns are innate and universal to nature- and paradoxically a combination of both.

        John Baez has a link on google to Vi Hart

        https://plus.google.com/117663015413546257905/posts?partnerid=ogpy0

        check it out for the fun of it it is long,.. end is best and she seems to think like I do and comments seem lame to me on her.

      • Orwin O'Dowd says:

        Life’s little ironies: Peter Woit’s notes on representation theory:

        http://www.math.columbia.edu/~woit/QM/qmbook.pdf

        He’s being more thorough than any of us can be (although I had flagged algebraic characters at the cutting edge), but not in a month of Sundays will he ever throw at it the kind of new, raw and controversial data and ideas that swim around here…

        Highlights: U(1) as charge – new on me.
        SO(3) – now when that splits into SO(1,3) and (3,1) you have the PIN groups, with a lead on neutrino chirality. That’s where the Grassmannian and the representations of time really matter.

  10. Lawrence B. Crowell says:

    I didn’t do lattice gauge work, but did Regge calculus.

    I am of the mind that a great idea will always beat a supercomputer. Supercomputers are great for modeling complex systems, or working on nonlinear problems (QCD or general relativity), or compiling and searching through vast amounts of data. Yet I think that nature or the universe is such that the great central ideas behind what we will learn in the future will be amenable to insight, imagination and the ability to ask questions. Computers do not ask questions.

    The push for ever faster computers that can store more data stems from a number or sources. Probably the first is profits. The market growth for that is huge, from keeping an information advantage over adversaries to the desire by people to send ever greater amounts of useless data around. As that continues it is likely we humans will become ever more integrated into a virtual reality domain that couples in closer to our senses and I think ultimately the brain itself.

    LC

  11. Donald DeGracia says:

    Mapping the anatomy of the brain at a specific level or levels of space and time will always be inherently incomplete because of the levels left out. Our existence as aware creatures depends not only on the microscopic fine structure of a single cell, not only the embedding of that cell in a local network, and several higher layers of networks, but of the whole brain embedded in a functioning body, that in turn is embedded in a society, that is embedded in the world. And the cell needs to have in it molecules and atoms, and all the smaller things you folks here study. A simulation of the cellular level of organization of the brain will not contain these elements that are present in our real functioning brains.

    Arbitrarily picking one, or even a handful, of the spatial and temporal levels of function and simulating it (them) will not capture the totality of the system. It is the whole system required for what we experience.

    Therefore, I predict that this will go the way of previous work and also not produce anything resembling real intelligence.

    Leibniz had an idea about these matters that he called a “monad”. Very few practicing scientists know this theory and it has been relegated to the dustbin of philosophy. But the monad theory addresses these issues head on by postulating an infinitely nested system, where at each level, the individual units act as holograms of the whole system. That is, each node, at each level of an infinite hierarchy reflects all other events in the universe. The quality of the reflections is variable in terms of resolving power. When the resolving power is lesser, then fine details somehow are lumped into aggregate structures that could, in principle, be decoded to reveal the fine details.

    This idea was far ahead of its time. But science and technology have come around to a level where we can begin to make sense of what Leibniz’ vision. Whether the system is infinite in nesting, for example, can intelligently be considered nowadays. But the overall model is of value.

    The quality we call “awareness” occurs at each level, and is the experiential aspect of the resolving power of the individual units.

    In this model, nothing causes awareness, it is axiomatically of the nature of the system.

    Leibniz spoke about man-made machines that lack this quality of the infinite nesting of monads and concluded they can never be aware in the fashion we are aware.

    He thought through these problems in a simpler time, when the key issues were more obvious and not so clouded by the details we know today. I think his work merits consideration in the present context.

    Thank you for considering my comments.

    • Stephen Crowley says:

      Donald, brilliant my friend, typed like someone who really gets it, and if it was read aloud, I would say it was spoken by someone who really gets it. How do I thumbs up this post and erase 99% of everything else on the internet? Kurzweil can put this in his pipe and smoke it.

      • Donald DeGracia says:

        Stephen, thank you for the kind comments. In this forum, I was not expecting such a reception. I do strongly recommend checking out Leibniz’ writings. He was a very insightful and clever person.

  12. Stephen, well stated by Donald… I too a fan of the infinitesimal as part of the big picture, thru a window darkly in the monad of Leibinz.
    I continue from the natural-unnatural blog thread questions there concerning biology and computation which relates to this issues of what sort of structure, mechanical or software, in the chaos of this internet, would in principle simulate intelligence- is it inevitable conscious arises everywhere if it seems other organic structures may according to a unified view of physics? That is the question to which we have to probe much deeper to begin to ask intelligently.

    We could of course say awareness would be a better linking of our vast amounts of data in whatever parallel processes and nodes- but would this make a higher sentience than that of any lesser organism than we have evolved to become? Can the knowing persist if it is part of the process of knowing in a stable healthy system?

    Sentience, Soul and Abstract Structures is my last post… What good is asking something like Jonathon did on generations if it is merely a lead to offer one’s one answer as a pet theory- I hope he listens to my answers in depth because as conscious as he is of the spirit of geometry and totality (Phi things as well as the term soul will put people off as sacred geometry) his imagination has not quite leaked out beyond the indefinite box to wider visions nor as with us all can we imagine yet what is needed just a little further beyond.

    Now Stephen, with all this physics in mind I get the impression the internet is there with free access by all and not the trolls at the toll bridges who in the end cannot answer our riddles.

Follow

Get every new post delivered to your Inbox.

Join 281 other followers

%d bloggers like this: