Wednesday, June 27, 2007
Inverting the research Piramid
Turning the search engine model upside down is a good start in more targeted research.
A search engine is based on power, raw power. Let's face it: there are millions of new pages created on the World Wide Web every week. No one can keep track of it all. Even major search engines like Google are only a small help.
The philosophy of a search engine is to gather information, as much information as possible, and then—based on advanced mathematical algorithms and data-mining capabilities—try to provide best bets to searchers. A graphical representation would be a triangle like an upside-down pyramid: information gathered at the entry points (top two angles) is slowly distilled down to get individualized results (the bottom angle).
The question is, is this the only way? What if I already know, more or less, where the information that I am looking for is stored? Many professionals know their industry well—through the magazines, the blogs, and even the forums that are relevant for their specific business or scientific interests.
Does such a professional need to use always a search engine? Is there not a different way to organize his or her information requirements? Could information be gathered in a much more selective way? Try inverting the search triangle mentioned previously, this time like a pyramid with the single angle at the top.
Industry professionals should then be able to get very selective and industry-specific searching environments, having the ability to experiment with them and readjust them to their requirements regularly. In a manner we would be moving away from the centralized searching offered by the search engine and toward a system that is decentralized and powerful in a very specific manner, without needing tremendous computing power every time.
A sort of "dashboard" would allow users to adjust their requirements from an industry-specific model to specific search and research requirements. Instead of handling millions of pages in a search, this system would handle a very targeted hundreds of thousands.
That there is a requirement for it is part of the reason why IBM has produced WebFountain. While it is obvious that IBM has a captive audience in all of its existing customers of on-demand computing, it has, again, followed the path of the search engines and missed an opportunity to develop what I have just described. WebFountain provides specific search environments, but they are based on the raw power of the system and are provided as an expensive add-on to an outsourcing and computing-on-demand contract.
There is a clear need for the specific search environments I describe, and I believe Intelliseek in the United States and DiceLaRed in Spain are the companies closest to suiting this need.
I encourage other members of the AlwaysOn Network and Zaibatsu to let me know what you think.
A search engine is based on power, raw power. Let's face it: there are millions of new pages created on the World Wide Web every week. No one can keep track of it all. Even major search engines like Google are only a small help.
The philosophy of a search engine is to gather information, as much information as possible, and then—based on advanced mathematical algorithms and data-mining capabilities—try to provide best bets to searchers. A graphical representation would be a triangle like an upside-down pyramid: information gathered at the entry points (top two angles) is slowly distilled down to get individualized results (the bottom angle).
The question is, is this the only way? What if I already know, more or less, where the information that I am looking for is stored? Many professionals know their industry well—through the magazines, the blogs, and even the forums that are relevant for their specific business or scientific interests.
Does such a professional need to use always a search engine? Is there not a different way to organize his or her information requirements? Could information be gathered in a much more selective way? Try inverting the search triangle mentioned previously, this time like a pyramid with the single angle at the top.
Industry professionals should then be able to get very selective and industry-specific searching environments, having the ability to experiment with them and readjust them to their requirements regularly. In a manner we would be moving away from the centralized searching offered by the search engine and toward a system that is decentralized and powerful in a very specific manner, without needing tremendous computing power every time.
A sort of "dashboard" would allow users to adjust their requirements from an industry-specific model to specific search and research requirements. Instead of handling millions of pages in a search, this system would handle a very targeted hundreds of thousands.
That there is a requirement for it is part of the reason why IBM has produced WebFountain. While it is obvious that IBM has a captive audience in all of its existing customers of on-demand computing, it has, again, followed the path of the search engines and missed an opportunity to develop what I have just described. WebFountain provides specific search environments, but they are based on the raw power of the system and are provided as an expensive add-on to an outsourcing and computing-on-demand contract.
There is a clear need for the specific search environments I describe, and I believe Intelliseek in the United States and DiceLaRed in Spain are the companies closest to suiting this need.
I encourage other members of the AlwaysOn Network and Zaibatsu to let me know what you think.
# posted by Rodolfo @ 3:05 PM 
