Harvard Business Publishing

This is the first time I have seen this web page. I am very iterestd in articles like this one.

It doesn't sound like you know what you're talking about...

Is this story a joke? Do you have any idea about what MapReduce actually is?
MapReduce is simply a framework for cluster computing. Also, you are aware that MapReduce has been in use for years, right? MapReduce is not new: http://labs.google.com/papers/mapreduce.html (from 2004).

Thanks for all of the feedback. Too often however, technologists mistake technology for products. I fully understand the technology behind MapReduce and true to the feedback, it is a “framework for cluster computing.” But, and this is a BIG but, the products that have been built around MapReduce are enabling brainlike intelligence and that is the point of this post.

On top of that, you must remember that the brain is really just a machine, albeit one of the most complex machines on the planet. So to say that my point is wrong just because MapReduce is old or just a clustering system is missing the point entirely. The point is that the brain is just a distributed processing system with MapReduce-like software enabling it.

These are the words of the Google folks, not mine: "[MapReduce] has been used across a wide range of domains within Google including: large-scale machine learning problems; clustering problems...; extracting data to produce reports of popular queries...;language model processing for statistical machine translation, and; large-scale graph computation." Even the most stogy neuroscientist would have no problem replacing Google, machine, and MapReduce, with different functions of the brain in the above sentence.

This type of broad-based cognitive functioning is enabled by distributed computing, combined with the ability to map and reduce large amounts data. This is what the brain does!

Best,

Jeff, the problem is not with the products. Maybe the point of this post, as you mean it, "the products that have been built around MapReduce are enabling brainlike intelligence". I'm not a brain scientist, so I cannot argue about brainlike intelligence, but if you replaced it by "the products that have been built by Google are enabling brainlike intelligence", there will be less questions.

MapReduce doesn't enable any intelligence. It's just a small piece of technology that improves scaling of processing. The real intelligence here is *processing*. In other words, map(data, function) — "function" here is the intelligence, not the "map" itself.

Take this sentence of your article:

"But Google is building a new secret weapon that has more to do with the brain than search. The effort is called MapReduce, a simple yet powerful software program that enables Google to use the Internet to think."

It's just factually wrong.

1) Google is not building the "weapon", they have already build it.
2) The "weapon" is not secret — everybody knows about it, there are other users of this technology other than Google, and you've cited the paper on MapReduce, and gave a link to it — how's that secret?
3) It doesn't enable Google to use Internet to think.

Your description of what MapReduce does is wrong:

"MapReduce does what our brains do all the time: It categorizes (Maps) key pieces of information, distributes it across its server farm of PCs, and then eliminates (Reduces) irrelevant data (computers--unlike MapReduce and the brain--soak in everything)."

"Map" is here not for categorization, and "Reduce" is not for eliminating of irrelevant data. Yes, the words (map and reduce) may be misleading. MapReduce doesn't do anything like this by itself.

Here's what it does (from the paper):

"MapReduce is a programming model and an associated implementation for processing and generating large data sets. Users specify a map function that processes a key/value pair to generate a set of intermediate key/value pairs, and a reduce function that merges all intermediate values associated with the same intermediate key. Many real world tasks are expressible in this model, as shown in the paper."

The following sentence is wrong:

"computers--unlike MapReduce and the brain--soak in everything"

I don't know how to explain this, but MapReduce soak in everything as well.

Thanks Dmitry for your thoughtful response and I do see where the problem is. I think we are talking about the same thing technology-wise, but you may be missing a piece of the brain science. The elegance behind the brain, it turns out, is really not that different than MapReduce. The brain (really the cerebral cortex) spends a good deal of energy on “scaling and processing.” This has historically been incredibly difficult to replicate. MapReduce’s innovation is that it mimics the brain in this functionality. And the “new secret” I was referring to was not MapReduce itself, but the fact that it was similar in scope to the functioning of the cerebral cortex.

That said, your points are well taken and we will make sure to be clearer next time around!

Jeff,

You should also be aware of the derivative software framework called Hadoop. Hadoop is similar to MR, and is in use today at Yahoo, among others. See here or here for more info.

Generally, organizations with huge clusters of computers and massive parallel data problems use some variant. It's interesting the parallel the general algorithm have with thinking, but it's not just Google's secret weapon, as some have pointed out.

best,
::t

I'm not a scientist nor do I stay up to date on the latest technologies. You can discuss the academics of underlying technologies, but at the end of the day Google is better at presenting the collective data, ideas and opinions on the internet in a logical form than anyone else.

In my opinion it's not what you know that matters, but what you do with your knowledge. Applying information technology to solve business processes or add value to people's lives is an entirely different discussion than the academics of any one technology.

Ward
Cusco, Peru
www.lifeinperu.com