Posts Tagged ‘analysis’

It’s very evident to me that businesses, organizations and individuals who don’t handle data well (i’ll define that shortly) don’t end up making any difference (traffic, profit, buzz…).

yeah, that’s probably not intellectual news to anyone.   really, though, how many people really handle data well?

Here are some common samples bad analysis, bad data, bad labeling, bad process:

  • VCs seriously consider 3 year pro-formas on businesses that have yet to produce or sell a single unit
  • Ad Agencies blatantly ignore sources of traffic when reporting to their clients
  • The whole media world pays attention to comscore, nielsen (and some even alexa!)
  • Product managers never track down baselines and expectations
  • Ad sales teams routinely ignore inventory levels
  • Marketers talk about “brand value”
  • dotcoms install 5 or 6 tracking mechanisms and never sync them
  • analysts/bi people start analysis with false assumptions or no assumptions
  • home buyers don’t calculate property taxes or relative market value of their home
  • employees generally don’t consider all implications of FSA and 401k contributions when consider real take home pay
  • employers evaluate employees on qualities and skills not results
  • traditional resumes feature dates and objectives not results and plans
  • dow = market to general public
  • subprime is word of the year
  • “backing into” a model is a well honed practice in most executive offices
  • Music labels pay attention to “money lost to piracy”

There are an infinite number of anecdotes on fishy data analysis.

For those that want actual facts – here’s how I know data analysis is a problem in industry and society:

Ok, ok.  I’ve done a good job of pointing out horrible data analysis and lots of fun factoids but I haven’t demonstrated why poor analysis diminishes opportunities.

First, let me explain my qualifications for “good analysis”:

  • data should be collected and analyzed in an appropriate timeframe (don’t take 10 years to graduate!)
  • Make a clear statement of analytic objective and methods is a must
  • The accuracy and depth of data and analysis should be relative to the importance of the subject matter
  • prediction of human behavior is impossible, avoid absolutists statements
  • explain relationships between variables, avoid overbearing causation arguments
  • check and recheck (1 set of eyes is not enough)
  • qualitative research should always accompany quantitative, vice versa
  • ask more questions

With some of those key statements established, i can now draw out why people and orgs miss out or flat out make huge mistakes so often.

[I will do so in a forthcoming post!]

Read Full Post »

Grid is here and it’s a game changer.  Not today, maybe not totally in 08, but certainly in the nearish future.

What is grid computing (cloud computing), you ask?  well, it’s lots of things.  Generally it refers to the idea that you can rent N number of cpu cycles to compute whatever you need.  Run websites, crunch datasets, run simulations, parse logs.  whatever you need to do, just rent the cycles to do it from grid computing providers, companies with excess cpu time or from your friendly neighborhood tech.

Grid is useful now because the tools to benefit from it are finally easy enough to generate adoption.  Amazon’s EC2 Cloud computing is amazing.  Really it is.  A webservice approach to setting up custom “nodes”.  Billed simply into accounts you probably have had for years.  Tons of documentation, samples, support developers… all there for you.

Yahoo just invest in Hadoop which is somewhat of grid computing.  Google is a gigantic grid computer system (use GWT to take some advantage of it!)  All available to Fortune 5, government, and YOU!

Technically, this matters because you can do a lot more when you do have to sweat the cycles.  Really.  if there’s no computational limit to what you are doing (other than can you afford it) all sorts of new services can be created.  New games, new investor tools, new education software, new advertising, new communications, new social networks.  Bandwidth was the first big damn to break.  With giant pipes readily available, we got to move away from text only experiences.  Look what’s resulted!?!  Computational power is another damn we’re breaking.  Retargeting of content, behavior analysis on the fly, improved AI…  all available to the common dev.  That’s huge.

At first I thought it would hurt hosting provides, hardware makers and so forth.  Actually though, i think it’s additive.  It’s yet another tool we can all use. It doesn’t replace always on, dedicated servers nor fast locked down storage.  It simply gives us lots of cycles as we need them to do interesting things.  And because I can’t see the future in any detail, I can’t make any claims about what it might do to existing industry and technology.

If you haven’t played with this stuff or even read about it, you need to.  It likely will be embedded in most online (and what isn’t online anymore?) within a decade.  web services and ajax was just the tip of this type of thinking.

Here’s what I want to do with cloud computing:

  • Find largest Mersenne Prime Number
  • Power my Decision Engine product (evolution of search engines to actually guide decisions)
  • Hook into ad servers to reforcast in realtime and retarget media based on behavior
  • Hook into a swarm of networked NXT bots to create social behavior across geography
  • fingerprint all YouTube videos and categorize based on transcripts and similarity scores (good for targeting ads or finding related media)
  • Create first homegrown weather forecasting simulation from Global models to Weather On the Ground. make freely available to all
  • Analyze social networks in real time
  • create a bot to play halo 3 for me all the time, but actually using the controller and data on SCREEN!
  • more more more


Read Full Post »