Posts Tagged ‘writing’

We have a problem.

As it stands now the present and near future of economic, social and cultural development primarily derives from computers and programming.   The algorithms already dominate our society – they run our politics, they run our financial system, they run our education, they run our entertainment, they run our healthcare.    The ubiquitous tracking of everything that can possible be tracked determined this current situation.   We must have programs to make things, to sell things, to exchange things.


The problem is not necessarily the algorithms or the computers themselves but the fact that so few people can program.    And why?   Programming Sucks.

Oh sure, for those that do program and enjoy it, it doesn’t suck. As Much.   But for the 99%+ of the world’s population that doesn’t program a computer to earn a living it’s a terrible endeavour.

Programming involves a complete abstraction away from the world and all surroundings.  Programming is disembodied – it is mostly a thought exercise mixed with some of the worst aspects of engineering.   Mathematics, especially the higher order really crazy stuff was long ago unapproachable and completely disembodied requiring no physical engineering or representation at all.  Programming, in most of its modern instances, consequences very far away from its creative behavior.  That is, in most modern system it takes days, weeks, months years to personally feel the results deeply of what you’ve built.    Programming is ruthless.  It’s unpredictable.   It’s 95% or more reinventing the wheel and configuring environments to even run the most basic program.  It’s all set up, not a lot of creation.   So few others understand it they can’t appreciate the craft during the act (only the output is appreciated counted in users and downloads).

There are a couple of reasons why this is the case – a few theoretical/natural limits and a few self-imposed, engineering and cultural issues.

First the engineering and cultural issues.   Programming languages and computers evolved rather clumsily built mostly by programmers for other programmers – not for the majority of humans.    There’s never been a requirement to make programming itself more humanistic, more embodied.    Looking back on the history of computers computing was done always in support of something else, not for its own sake.   It was done to Solve Problems.   As long as the computing device and program solved the problem the objective was met.   Even the early computer companies famously thought it was silly to think everyone one day might actually use a personal computer.   And now we’re at a potentially more devastating standstill – it’s absurd to most people to think everyone might actually need to program.    I’ll return to these issues.

Second the natural limits of computation make for a very severe situation.   There are simply things that are non-computable.   That is, we can’t solve them.   Sometimes we can PROVE we can’t solve them but that doesn’t get us any closer to solving some things.    This is sometimes called the Halting Problem.  The idea is basically that for a sufficiently complex program you can’t predict whether the program will halt or not.   The implication is simply you must run the program and see if it halts.  Again, complexity is the key here.  If these are relatively small, fast programs with a less than infinite number of possible outcomes then you can simply run the program across all possible inputs and outputs.   Problem is… very few programs are that simple and certainly not any of the ones that recommend products to you, trade your money on wall street, or help doctors figure out what’s going on in your body.


This is a VERY BIG DEAL.    Think about it.   We deploy millions of programs a day with completely non-deterministic, unpredictable outcomes.  Sure we do lots of quality assurance and we test everything we can and we simulate and we have lots of mathematics and experience that helps us grow confident… but when you get down to it, we simply don’t know if any given complex program has some horrible bug in it.

This issue rears its head an infinite number of times a day.   If you’ve ever been mad at MS Word for screwing up your bullet points or your browser stops rendering a page or your internet doesn’t work or your computer freezes… this is what’s going on.  All of these things are complex programs interacting with other programs and all of them have millions (give or take millions) of bugs in them.  Add to it that all of these things are mutable bits on your computer that viruses or hardware issues can manipulate (you can’t be sure the program you bought is the program you currently run) and you can see how things quickly escape our abilities to control.

This is devastating for the exercise of programming.  Computer scientists have invented a myriad of ways to temper the reality of the halting problem.   Most of these management techniques makes programming even more mysteries and challenging due to the imposition of even more rules that must be learned and maintained.   Unlike music and writing and art and furniture making and fashion we EXPECT and NEED computers to do exactly what we program them to do.   Most of the other stuff humans do and create is just fine if it sort of works.  It still has value.  Programs that are too erratic or worse, catastrophic, are not only not valuable we want to eliminate them from the earth.   We probably destroy some 95%+ of the programs we write.

The craft of programming is at odds with its natural limits.   Our expectations and thus the tools we craft to perform program conflict with the actuality.  Our use of programs exceeds their possibilities.

And this really isn’t due to computers or programming, but something more fundamental: complexity and prediction.    Even as our science shows us more and more that prediction is an illusion our demands of technology and business and media run counter.    This fundamental clash manifests itself in programming, programming languages, the hardware of computers, the culture of programming.  It is at odds with itself and in being so conflicted is unapproachable to those that don’t have ability to stare maddeningly into a screen flickering with millions of unknown rules and bugs.   Mastery is barely achievable except for a rare few.   And without mastery enjoyment rarely comes – the sort of enjoyment that can sustain someones attention long enough to do something significant.

I’ve thought long and hard about how to improve the craft of programming.   I’ve programmed a lot, lead a lot of programming efforts, delivered a lot of software, scrapped a lot more.  I’ve worked in 10+ languages.  I’ve studied mathematics and logic and computer science and philosophy.  I’ve worked with the greatest computer scientists.  I’ve worked with amazing business people and artists and mathematicians.   I’ve built systems large and small in many different categories.  In short, I’ve yet to find a situation in which programming wasn’t a major barrier to progress and thinking.

The solution isn’t in programming languages and in our computers.  It’s not about Code.org and trying to get more kids into our existing paradigm. This isn’t an awareness or interest problem.   The solution involves our goals and expectations.

We must stop trying to solve every problem perfectly.  We must stop trying to predict everything.   We must stop pursuing The Answer, as if it actually exists.  We must stop trying to optimize everything for speed and precision and accuracy. And we must stop applying computerized techniques to every single category of activity – at least in a way where we expect the computer to forever to the work.

We must create art.  Programming is art.  It is full of accidents and confusions and inconsistencies.   We must turn it back to an analog experience rather than a conflicted digital.    Use programming to explore and narrate and experiment rather than answer and define and calculate.

The tools that spring from those objectives will be more human.  More people will be able to participate.  We will make more approachable programs and languages and businesses.

In the end our problem with programming is one of relation – we’re either relating more or less to the world around us and as computers grow in numbers and integration we need to be able to commune, shape and relate to them.

Read Full Post »

The Point

Everything is a pattern and connected to other patterns.   The variety of struggles, wars, businesses, animal evolution, ecology, cosmological change – all are encompassed by the passive and active identification and exploitation of changes in patterns.

What is Pattern

Patterns are thought of in a variety of ways – a collection of data points, pictures, bits and bytes, tiling.   All of the common sense notions can be mapped to the abstract notion of a graph or network of nodes and their connections, edges.   It is not important, for the sake of the early points of this essay, to worry to much about the concept of a graph or network or its mathematical or epistemological construction.   The common sense ideas that might come to mind should suffice – everything is a pattern connected to other patterns. E.g. cells are connected to other cells sometimes grouped into organs connected to other organs sometimes grouped into creatures connected to other creatures.


As can be imagined the universe has a practically infinite number of methods of pattern identification and exploitation. Darwinian evolution is one such example of a passive pattern identification and exploration method. The basic idea behind it is generational variance with selection by consequences. Genetics combined with behavior within environments encompass various strategies emergent within organisms which either hinder or improve the strategies chance of survival. Broken down and perhaps too simplistically an organism (or collection of organisms or raw genetic material) must be able to identify threats, energy sources and replication opportunities and exploit these identifications better than the competition.   This is a passive process overall because the source of identification and exploitation is not built in to the pattern selected, it is emergent from the process of evolution. On the other hand sub processes within the organism (object of pattern were considering here) can be active – such as in the case of the processing of an energy source (eating and digestion and metabolism).

Other passive pattern processes include the effects of gravity on solar systems and celestial bodies on down to their effects on planetary ocean tides and other phenomena.   Here it is harder to spot what is the identification aspect?   One must abandon the Newtonian concept and focus on relativity where gravity is the name of the changes to the geometry of spacetime.   What is identified is the geometry and different phenomena exploit different aspects of the resulting geometry.   Orbits form around a sun because of the suns dominance in the effect on the geometry and the result can be exploited by planets that form with the right materials and fall into just the right orbit to be heated just right to create oceans gurgling up organisms and so on.   It is all completely passive – at least with our current notion of how life my have formed on this planet. It is not hard to imagine based on our current technology how we might create organic life forms by exploiting identified patterns of chemistry and physics.

In similar ways the trajectory of artistic movements can be painted within this patterned theory.   Painting is an active process of identifying form, light, composition, materials and exploiting their interplay to represent, misrepresent or simply present pattern.   The art market is an active process of identifying valuable concepts or artists or ideas and exploiting them before mimicry or other processes over exploit them until the value of novelty or prestige is nullified.

Language and linguistics are the identification and exploitations of symbols (sounds, letters, words, grammars) that carry meaning (the meaning being built up through association (pattern matching) to other patterns in the world (behavior, reinforcers, etc).   Religion, by the organizers, is the active identification and exploitation of imagery, language, story, tradition, and habits that maintain devotional and evangelical patterns. Religion, by the practitioner, can be active and passive maintenance of those patterns. Business and commerce is the active (sometimes passive) identification and exploitation of efficient and inefficient patterns of resource availability, behavior and rules (asset movement, current social values, natural resources, laws, communication medium, etc).

There is not a category of inquiry or phenomena that can escape this analysis.   Not because the analysis is so comprehensive but because pattern is all there is. Even the definition and articulation of this pattern theory is simply a pattern itself which only carries meaning (and value) because of the connection to other patterns (linear literary form, English, grammar, word processing programs, blogging, the Web, dictionaries).

Mathematics and Computation

It should be of little surprise that mathematics and computation forms the basis of so much of our experience now.   If pattern is everything and all patterns are in a competition it does make some common sense that efficient pattern translation and processing would arise as a dominant concept, at least in some localized regions of existence.

Mathematics effectiveness in a variety of situations/contexts (pattern processing) is likely tied to its more general, albeit often obtuse and very abstracted, ability to identify and exploit patterns across a great deal of categories.   And yet, we’ve found that mathematics is likely NOT THE END GAME. As if anything could be the end game.   Mathematics’ own generalness (which we could read as reductionist and lack of full fidelity of patterns) does it in – the proof of incompleteness showed that mathematics itself is a pattern of patterns that cannot encode all patterns. Said differently – mathematics incompleteness necessarily means that some patterns cannot be discovered nor encoded by the process of mathematics.   This is not a hard meta-physical concept. Incompleteness merely means that even for formal systems such as regular old arithmetic there are statements (theorems) where the logical truth or falsity cannot be established. Proofs are also patterns to be identified and exploited (is this not what pure mathematics is!) and yet we know, because of proof, that we will always have patterns, called theorems, that will not have a proof.   Lacking a proof for a theorem doesn’t mean we can’t use the theorem, it just means we can’t count on the theorem to prove another theorem. i.e. we won’t be doing mathematics with it.   It is still a pattern, like any sentence or painting or concept.


The effectiveness of mathematics is its ROBUSTNESS. Robustness (a term I borrow from William Wimsatt) is the feature of a pattern that when it is processed from multiple other perspectives (patterns) the inspected pattern maintains its overall shape.   Some patterns maintain their shape only within a single or limited perspective – all second order and higher effects are like this. That is, anything that isn’t fundamental is of some order of magnitude less robust that things that are.   Spacetime geometry seems to be highly robust as a pattern of existential organization.   Effect carrying ether, as proposed more than 100 years ago, is not.   Individual artworks are not robust – they appear different to any different perspective. Color as commonly described is not robust.   Wavelength is.

While much of mathematics is highly robust or rather describes very robust patterns it is not the most robust pattern of patterns of all. We do not and likely won’t ever know the most robust pattern of all but we do have a framework for identifying and exploiting patterns more and more efficiently – COMPUTATION.

Computation, by itself. 

What is computation?

It has meant many things over the last 150 years.   Here defined it is simply patterns interacting with other patterns.   By that definition it probably seems like a bit of a cheat to define the most robust pattern of patterns we’ve found to be patterns interacting with other patterns. However, it cannot be otherwise. Only a completely non-reductive concept would fit the necessity of robustness.   The nuance of computation is that there are more or less universal computations.   The ultimate robust pattern of patterns would be a truly universal-universal computer that could compute anything, not just what is computable.   The real numbers are not computable, the integers are.   A “universal computer” described by today’s computer science is a program/computer that can compute all computable things. So a universal computer can compute the integers but cannot compute the real numbers (pi, e, square root of 2). We can prove this and have (the halting problem, incompleteness, set theory….).   So we’re not at a completely loss of interpreting patterns of real numbers (irrational numbers in particular). We can and do compute with pi and e and square root millions of times a second.   In fact, this is the key point.   Computation, as informed by mathematics, allows us to identify and exploit patterns far more than any other apparatus humans have devised.   However, as one would expect, the universe itself computes and computes itself.   It also has no problem identifying and exploiting patterns of all infinitude of types.

Universal Computation

So is the universe using different computation than we are? Yes and no.   We haven’t discovered all the techniques of computation at play. We never will – it’s a deep well and new approaches are created constantly by the universe. But we now have unlocked the strange loopiness of it all.   We have uncovered Turing machines and other abstractions that allow us to use English-like constructs to write programs that get translated into bits for logic gates in parallel to compute and generate solutions to math problems, create visualizations, search endless data, write other programs, produce self replicating machines, figure out interesting 3D printer designs, simulate markets, generate virtual and mixed realities and anything else we or the machines think up.

What lies beneath this all though is this very abstract yet simple concept of networks.   Nodes and edges. The mathematics and algorithms of networks.   Pure relation between things. Out of the simple connection of things from things arise all the other phenomena we experience.   The network is limitless – it imposes no guardrails to what can or can’t happen. That it is a network does explain and impose why all possibilities exhibit as they do and the relative emergent levels of phenomena and experience.

The computation of pure relation is ideal.   It only supersedes (makes sense to really consider) the value of reductionist modes of analysis, creation and pattern processing when the alternative pattern processing is not sufficient in accuracy and/or has become sufficiently inefficient to provide relative value for it’s reduction.   That is, a model of the world or a given situation is only as value as it doesn’t overly sacrifice accuracy too much for efficiency.   It turns out for most day to day situations Newtonian physics suffices.

What Next

we’ve arrived at a point in discovery and creation where the machines and machine-human-earth combinations are venturing into virtual, mixed and alternate realities that current typical modes of investigation (pattern recognition and exploitation) are not sufficient. The large hadron collider is an example and less an extreme example than it was before. The patterns we want to understand and exploit – the quantum and the near the speed of light and the unimaginably large (the entire web index with self driving cars etc) – are of such a different magnitude and kind.   Then when we’ve barely scratched the surface there we get holograms and mixed reality which will create it’s own web and it’s own physical systems as rich and confusing as anything we have now. Who can even keep track of the variety of culture and being and commerce and knowledge in something such as Minecraft? (and if we can’t keep track (pattern identify) how can we exploit (control, use, attach to other concepts…)?

The pace of creation and discovery will never be less in this local region of spacetime.   While it may not be our goal it is our unavoidable fate (yes we that’s a scary word) to continue to compute and have a more computational approach to existence – the identification and exploitation of patterns by other patterns seems to carry this self-reinforcing loop of recursion and the need of ever more clarifying tools of inspection that need more impressive means of inspecting themselves…   everything in existence replicates passively or actively and at a critical level/amount of interconnectivity (complexity, patterns connected to patterns) self inspection (reasoning, introspection, analysis, recursion) becomes necessary to advance to the next generation (explore exploitation strategies).

Beyond robotics and 3d printing and self-replicating and evolutionary programs the key pattern processing concept humans will need is a biological approach to reasoning about programs/computation.   Biology is a way of reasoning that attempts to classify patterns by similar behavior/configurations/features.   And in those similarities find ways to relate things (sexually=replication, metabolism=Energy processing, etc).   It is necessarily both reductionist, in its approach to categorize, and anti-reductionist in its approach to look at everything anew. Programs / computers escape our human (and theoretical) ability to understand them and yet we need some way to make progress if we, ourselves, are to persist along side them.

And So.

It’s quite possible this entire train of synthesis is a justification for my own approach to life and my existence. And this would be consistent with my above claims.   I can’t do anything about the fact that my view is entirely biased by my own existence as a pattern made of patterns of patterns all in the lineage of humans emerged from hominids and so on all the way down to whatever ignited patterns of life on earth.

I could be completely wrong. Perhaps some other way of synthesizing existence all the way up and down is right. Perhaps there’s no universal way of looking at it. Though it seems highly unlikely/very strange to me that patterns at one level or in one perspective couldn’t be analyzed abstractly and apply across and up and down.   And that the very idea itself suggests patterns of pattern synthesis is fundamental strikes me as much more sensible, useful and worth pursuing than anything else we’ve uncovered and cataloged to date.

Read Full Post »