Feeds:
Posts
Comments

Posts Tagged ‘information’

A variety of thinkers and resources seem to converge on some fundamental ideas around existence, knowledge, perception, learning and computation.   (Perhaps I have a confirmation bias and have only found what I was primed to find).

 

Kurt Godel articulated and proved what I believe to be the most fundamental idea of all, the Incompleteness Theorem.   This theorem along with analog variants in the Halting Problem and other aspects of complexity theory provides us the notion that there is a formal limit to what we can know.   And by “to know” I mean it in the Leibnizen sense of perfect knowledge (scientific fact with logical proof, total knowledge).   Incompleteness tells us even with highly abstract, specialized formal systems there will always be some statement WITHIN that system that is true but cannot be proved. This is fundamental.

 

It means that no matter how much mathematical or computational or systematic logic we work out in the world there are just some statements/facts/ideas that are true but cannot be proven to be true.   As the name of the theorem suggests, though it’s mathematical meaning isn’t quite this, our effort in formalizing knowledge will remain incomplete.   There’s always something just out of reach.

 

It is also a strange fact that one can prove incompleteness of a system and yet not prove trivial statements within these incomplete formal systems.

 

Godel’s proof and approach to figuring this out is based on very clever re-encoding of formal systems laid out by Betrand Russell and A Whitehead.   This re-encoding of the symbols of math and language has been another fundamental thread we find through out human history.   One of the more modern thinkers that goes very deep into this symbolic aspect of thinking is Douglas Hofstadter, a great writer and gifted computer and cognitive scientist.   It should come as no surprise that Hofstadter found inspiration in Godel, as so many have. Hofstadter has spent a great many words on the idea of strange loops/self-reference and re-encodings of self-referential systems/ideas.

 

But before the 20th century Leibniz and many other philosophical, artistic, and mathematical thinkers had already started laying the groundwork around the idea that thinking (and computation) is a building up of symbols and associations between symbols.   Of course, probably most famously was Descartes in coining “I think, therefore I am.”   This is a deliciously self-referential, symbolic expression that you could spend centuries on. (and we have!)

 

Art’s “progression” has shown that we do indeed tend to express ourselves symbolically. It was only in more modern times when “abstract art” became popular that artist began to specifically avoid overt representation via more or less realistic symbols.   Though this obsession with abstraction turns out to be damn near impossible to pull off, as Robert Irwin from 1960 on demonstrated with his conditional art.   In his more prominent works he did almost the minimal gesture to an environment (a wall, room, canvas) and found that almost no matter what, human perception still sought and found symbols within the slightest gesture.   He continues to this day to produce conditional art that seeks to have pure perception without symbolic overtones at the core of what he does. Finding that it’s impossible seems, to me, to be line with Godel and Leibniz and so many other thinkers.

 

Wittgenstein is probably the most extreme example of finding that we simply can’t make sense of many things, really, in a philosophical or logical sense by saying or writing ideas.   Literally “one must be silent.”   This is a very crude reading and interpretation of Wittgenstein and not necessarily a thread he carries throughout his works but again it strikes me as being in line with the idea of incompleteness and certainly in line with Robert Irwin. Irwin, again no surprise, spent a good deal time studying Wittgenstein and even composed many thoughts about where he agreed or disagreed with Wittgenstein.   My personal interpretation is that Irwin has done a very good empirical job of demonstrating a lot of Wittgensteinien ideas. Whether that certifies any of it as the truth is an open question. Though I would argue that saying/writing things is also symbolic and picture-driven so I don’t think there’s as clear a line as Wittgenstein drew.   As an example, Tupper’s Formula is an insanely loopy mathematical function that draws a graph of itself.

 

Wolfram brings us a more modern slant in the Principle of Computational Irreducibility.   Basically it’s the idea that any system with more than very simple behavior is not reducible to some theory, formula or program that can predict it. The best we could do in trying to fully know a complex system is to watch it evolve in all its aspects.   This is sort of a reformulation of the halting problem in such a way that we might more easily imagine other systems beholden to this reality.   The odd facet of such a principle is that one cannot really prove with any reliability which systems are computational irreducible.   (P vs NP, etc problems in computer science are akin to this).

 

Chaitin, C. Shannon, Aaronson, Philip Glass, Max Richter, Brian Eno and many others also link into this train of thought….

 

Why do I think these threads of thought above (and many others I omit right now) matter at all?

 

Nothing less than everything.   The incompleteness or irreducibility or undecidability of complex systems (and even seemingly very simple things are often far more complex than we imagine!) is the fundamental feature of existence that suggests why, when there is something, there’s something rather than nothing. For there to be ANYTHING there must be something outside of full description. This is the struggle.   If existence were reducible to a full description there would be no end to that reduction until there literally was nothing.

 

Weirder, perhaps still, is the idea is the Principal of Computational Equivalence and Computational Universality.   Basically any system that can compute universally can emulate any other universal computer.   There are metaphysical implications here that if I’m being incredibly brash suggest that anything complex enough can and/is effectively anything else that is complex.   Again tied to the previous paragraph of thought I suggest that if there’s anything at all, everything is everything else.   This is NOT an original thought nor is it as easily dismissed as whacky weirdo thinking.   (Here’s a biological account of this thinking from someone that isn’t an old dead philosopher…)

 

On a more pragmatic level I believe the consequences of irreducibility suggest why computers and animals (any complex systems) learn the way they learn.   Because there is no possible way to have perfect knowledge complex systems can only learn based on versions of Probably Approximately Correct (Operant Conditioning, Neural Networks, Supervised Learning, etc are all analytic and/or empirical models of learning that suggest complex systems learn through associations rather than executing systematic, formalized, complete knowledge)   Our use of symbolics to think is a result of irreducibility.   Lacking infinite energy to chase the irreducible, symbolics (probably approximately correct representations) must be used by complex systems to learn anything at all.   (this essay is NOT a proof of this, this is just some thoughts, unoriginal ones, that I’m putting out to prime myself to actually draw out empirical or theoretical evidence that this is right…)

 

A final implication to draw out is that of languages and specifically of computer languages.   To solve ever more interesting and useful problems and acquire more knowledge (of an endless growing reservoir of knowledge) our computer languages (languages of thought) must become more and more rich symbolically.   Our computers, while we already make them emulate our more rich symbolic thinking, need to have symbolics more deeply embedded in their basic operations.   This is already the trend in all these large clusters powering the internet and the most popular software.

 

As a delightful concluding, yet open unoriginal thought from this book by Flusser comes to mind…   Does Writing Have a Future suggests that ever more rich symbolics than the centuries old mode of writing and reading will not only be desired but inevitable as we attempt to communicate in more vast networks. (which, won’t surprising, is very self-referential if you extend the thought to an idea of “computing with pictures” which really isn’t different than computing with words or other representations of bits that represent other representation of bits…)   I suppose all of this comes down to seeing which symbolic prove to be more efficient in the total scope of computation.   And whatever interpretation we assign to efficient is, by the very theme of this essay, at best, an approximation.

Read Full Post »

NOTE: this is NOT a mathematical proof, a formal logic paper, nor even a science paper.  It’s a blog post that contains interpretive statements and some shortcuts to get to the point.  Maybe not even the point (s) I meant to make.  also, i’m sure there are typos.

aka a Story.

My mostly-borrowed thesis: Everything is Information.

Various smart folks have put forward this basic theory.   And I’ve personally come to believe it as truth.

Seth Lloyd put this basic theory forward in a clear way for a popular audience in his book “Programming the Universe”.

The universe is made of bits.  Every molecule, atom, and elementary particle registers bits of information.  Every interaction between those pieces of the universe processes that information by altering those bits. (page 3, Introduction)

Lloyd proceeds to draw out the universe as a computer paradigm and make a compelling case that everything is just information processing.   It’s a paradigm many others have proposed but I really like the straight-forwardness of of Lloyd’s book.

Now I can’t prove his theory or this entire thesis that Everything is Information.  I think Lloyd and others have done a really good job making a case for this view.   I’m going to essentially treat it as an axiom and develop a train of thought from there.  In the end of my explorations I’m led to a somewhat less borrowed thesis.

Art (and in particular STORY) is the most effective way humans can understand the universe and thrive

I can’t prove this either but why not shine a light on some data, some ideas, some commentary to perhaps make it easier to engage with this theory?

This thesis results from following a common thread to responses to questions like:

  • What is a thought?
  • Who am I?
  • What is behavior?  where does it come from?
  • what is moral?  what is a law? what do we value?
  • what is computation?  what is a general computer?
  • is the universe/multiverse a computer?
  • how did it all begin?  how does it all end?
  • why do people laugh? what is humor?
  • what is art?  why is some art good and other bad?
  • what are forces?  what is DNA in the abstract?
  • what is mathematics?
  • what is language?  communication?
  • what is time?  what is space?  what is motion?  what is change?
  • what is death? what is life?
  • what is love?  is love just a word or a real thing?

There’s certainly a large body of work (UNDERSTATEMENT!) attempting to answer these questions rigorously and thoroughly.    By my interpretation of the work that I can actually consume, process and synthesize it all leads back to the kernel that the most fundamental concepts are information and the processing of information.   Everything is information, nothing is information.  A bit.  0.  1.  Infinity. Blackholes.  Planets.  People. DNA. RNA. Animals. Humans.  Language. Emotions. Behavior. Math. Love. Computers. Paintings. Books. Bosons. Time. Space. Existence.  Non Existence.

What is is information.  What happens is processing information aka computation.

Humans are a specific class of configurations of information.   Survival is maintaining this class of configurations throughout processing.  Evolution is the transformation of this class of configurations of information.   Understanding is the processing capability to be aware of information configuration and processing (this is so strange loopy meta like).  Thriving is a human ideal/feeling (also information configuration) of not merely surviving (passing genes on) but of actually playing a material and unique part of processing information.

What is Information then?

Seems to be a basic question to ask.

To be sure, this word information in communication theory relates not so much to what you do say, as to what you could say.  That is, information is a measure of one’s freedom of choice when one selects a message.

This comes from Warren Weaver’s introduction to Weaver and Shannon’s “The Mathematical Theory of Communication.”   This is a classic, the classic, book on information theory.  It is a good place to start even though the language is somewhat anthropological.

I take the above quote in a broader sense that information is a measure of anything’s freedom of choice to be something else, to interact with other information.  Everything has infinite freedom.  Nothing has infinite freedom.   All the various “things” or configurations of bits into bytes into megabytes and so has various measure of potential to be something/anything.

Whoa.  That’s a mouthful of abstraction and ambiguity.  Such is the danger of trying to talk about these topics!

[Remarkably reviewing entries on Wikipedia for Information yield a pretty confusing set of paths to explore the basic idea of information.  WolframAlpha yields a variety of definitions, usage patterns and related terms that also lead in a wide variety of directions and abstractions.  And perhaps, more interestingly, the choice was made to map the basic query “information” to pretty much EVERYTHING in WolframAlpha.]

The smallest amount of information is a bit.  a 1 or a 0.   that can be processed as open or shut, on or off, charge or no charge, etc.   Put more bits together and things get interesting quickly.  two bits and you get 4 numbers, little words, on, off, sort of on, sort of off and so on.   You can build up the multiverse from this.   You can write configurations of information that process other information aka  “programs”.  So the universe has a very large measure of information – lots of freedom of choice to configure bits.

And a little tangent here… don’t you need another concept “energy” that gives you the fuel to process information.  Um, if you need that definition you can use it.  It’s really just a short cut to get around defining everything in terms of information.   e.g. how much energy a system has is just information about the rules for processing information.

Which then leads to wonder why there seem to be specific rules (information) about how to process information that give us this universe we experience.  It’s not at all clear that this is true in the universe – that there are fundamental rules that cannot be different.   The universe (this specific configuration of information) may have rules that it probabilistically are most likely to play out, but there’s not a requirement in the space of all possibilities.

I have to stop this train before it becomes a complete paper / book / library unto itself.   Wolfram, Lloyd, Shannon, Chaitin, Wheeler, Deustch and many others go very in depth about this stuff.

It’s unlikely I’ve convinced you of Wheeler’s premise “it from bit” but hopefully there’s some understanding of how I interpret things.

What is Information Processing? What is Computation?

Well, in short, it’s the transformation of information configurations into other information configurations.   Oh, sure, we can pick this a part and try to get more rigorous, which again, I’ll just refer folks to the smart people better able to draw all that out.

Processing could be random, a computation, simply letting time pass, anything really.

Computation is a bit more specific but still nebulous.   Computation is a refinement of the general processing in the form of function or a program or an algorithm – a set of instructions or rules by which the processing occurs.   I think it’s good to have this really abstract thing called processing and something more specific like computation because when you dig deep into things like computability you need these distinctions.   Not all processing is computable processing.

However, in general I don’t really make much of a distinction going forward.

Now to make sense of any of this and make progress we have to tackle the universe of information configurations and how they come to be and how we figure them out.

What is Exploring The Space of Possibilities and Why Does That Matter

The universe is always computing.  It’s exploring all possible configurations of information.   We experience and/or observe just a tiny tiny bit of these configurations.

Computing/processing (observing, understanding, modeling, sharing) ALL information configurations takes more time and energy than any of us have. Heck, processing even a small portion of information takes more time and energy than we have.  (wait, pause!   by limits time and energy… I mean this current configuration of information we are in the form of cells, organs, brains, humans has instructions to transform into other information aka we die.)

The survival of humanity and of an individual depends on exploring ways of avoiding extinction in the face of information processes that change us (kill us, destroy the genetic code, etc).

If one’s goal beyond survival is to live well (thrive) by whatever definitions we concoct then we also need to explore the universe of possibilities at that level as well.  And yes, I believe, our class of configurations, humans, has some embedded and learned processing instructions to do this.  Perhaps it wasn’t always embedded but the process of evolution (or whatever other processing model is in place) seems to have selected a class of configurations that tries to thrive over those that just maintain the gene code.

So.

There have been attempts to explain and interpret EVERYTHING through mathematics, physics, computer science, philosophy, religion, and so forth.   All of these attempts are models of how it all works.  Models of information and processing information that are more or less useful for figuring out ways to survive (and then to thrive).  These are narratives or stories.  Some more “formal” and “coherent” or “logically consistent” than others i.e. less open to interpretation and varied application of those interpretations.

What becomes apparent as you dig into each of these narratives and their connections to each other is that to actual make use of these narratives in our own lives consumes considerable amount of energy – more than our instruction sets provide.   In short, you could not actually get through a day if all you did was try to use “math” to navigate life.  Mathematical interpretation of all this information adds a layer of information that becomes all consuming to other forms of information processing that actually keep you alive much provide understanding.

Cutting to the chase, which is so hard to do, is that there are infinite number of information processing methods to gain understanding at work all the time.   Math is one approach (well, it’s infinitely rich as well).   Chemistry is another approach.  and so on.   All are universal processors – given enough time/energy they will explore the right possibilities.

And here we get to the BIG THESIS is that ART and STORY are the most efficient ways to explore the right information processing for humankind to improve chances of survival of the species and of an individual.

How Does Art, Story Compute and Explore the Right Possibilities more Efficiently

For whatever reason human nervous systems seem to be big fat pattern recognizers.   That is they “see” patterns and change information configurations (behave) based on patterns.   Successive exposure to the same pattern or similar patterns tends to reinforce specific behavior aka learning.  (see experimental analysis of behavior for things like matching relation, etc. and various other learning theory and neuroscience material).

Learning is essential to avoiding “destructive” information configuration transformations (ya know, death).

So this thesis comes down to figuring out which ways of processing the universe teach the species (and its individuals) efficiently.   

And this is where this essay has no ability to prove anything with rigor.   That said, here goes.

Efficient learning involves efficient presentation of stimuli and efficient processing of that stimuli.   In other words, to effectively teach someone you have to be able to communicate information with them in such a way that they can consume it, process it and learn from it with the limited time and energy they have to avoid destruction.   There are some basic survival things “learned” in the gene code… various fixed action patterns like suckling and crying that get us going, but after that learning has to take pretty quickly to avoid the million different ways we can die at any given moment.

Now, before we get all crazy, let’s consider that humankind very much could have a different strategy for survival.  But the fact is our current configuration is such that we take 9 months to bake in the womb, we come out needing lots of help and have a very long rearing stage while our brains and bodies grow and get to the point where we can pass on the gene code (can make eggs and sperm and mate).   Having a person live this long and to select a viable mate makes learning some complicated stuff very quickly essential.   And if you keep thinking about all this you end up looping in about did big brains create the need to learn or did stimuli start evolving brains (bad example) and all sorts of other statements we can never verify.

So here we are with this species.  Over the centuries we’ve taught generation after generation how to survive and then how to contribute to the survival of the species. Which, to me, seems to rely on convincing each other to not just survive but to thrive so we’re more attractive to each other and all feel like living long enough to be fruitful and multiply.

What appears to be mostly true from history is that our primary way of teaching is through narrative.  We concoct stories that are devoid of formal specifics and instead have some memorable themes, lessons and characters – you know, patterns we can interpret in a wide variety of contexts.

These stories come in the form of fables, religion, traditions, paintings and what not. ( I am not suggesting MEMEs. )

Stories seem to be really robust information packets.   They can be poorly told and retain information value.  They can carry on through various mediums.   They are primitive packets of human information that survive generational death.

Formal mathematics, science texts, and what not are very dense information packets requiring very specific processing capability (a long time spent learning math!).

In essence stories help us avoid dying due Computational Irreducibility.   Most things we experience, see are computationally irreducible.   That is, to fully understand them would take forever and infinite energy.  Stories provide a description of how the world works that our pattern recognition systems can story up a bunch of stories that help us react without needing complete knowledge.  Stories are usually comprised of metaphors or rather we are good at using stories metaphorically to expand their utility.   Bears eat people is equivalent to Big Brown things with Claws eat People and so on.  (worth reading is Metaphors We Live By and responses like this)

It’s quite possible that with modern computers we’ll escape our current configuration computing limitations and we can describe the universe and the world around us with ever more precision and have enough time to not just live but thrive.

As it stands now, we’re still a world that relies on the telling of stories.

Our businesses need PR and business plans.   Our politicians need platforms and slogans.   Our kids need fables.   Our families need traditions.  Our economy needs advertising.

If we could simply process ALL INFORMATION we wouldn’t need short hand or interpretive information packets.

What Are The Implications

I think if we eliminate the need for story we’re not going to at all resemble this information configuration known as human.   It’s neither bad nor good.  Just different.

I think Story = Human.

I think we’re seeing, in some aspects of culture, the erosion of story and thus humanity.  Facebook and twitter are more and more turning the daily experience into more and more specific, formal bytes of what’s going on.   It’s quite possible that as web content gets more algorithmically generated we’ll just use algorithms to interpret it and as we get our phones and smart devices to do more and more stuff for us we’ll probably lose the ability and/or the need to tell stories and we won’t know the difference or care.

Humans aren’t efficient by very many measures.   What we’re efficient at is telling and interpreting stories.  This may not turn out to be a good ability for long term survival.   I don’t even know of species survival is a good thing.

I do think everything is information and we’re part of that everything and that stories are a nifty little thing in the configuration of all things.  and that of all the big questions I’ve chased down in life almost all of them have the best answers found in a story.   It is a tale told by an idiot perhaps…..

Read Full Post »