After 911 AltaVista scored some rare Brownie Points against Google, in the press, because Google didn't have news search, but AV did, via Moreover. Google News was built largely as a result of 911.
It proved that Moreover was a news search engine, but it was too late.
Our PR company had told us that 'search is dead' and without a revenue model for search engines, there was pressure from all sides to make Moreover something else - which resulted in all sorts of convoluted bullshit and meant that Moreover never had decent technology for full-text search.
Eppur Si Muove.
Technorati is in the same boat, there is probably a great deal of pressure not to call it a blog search engine, and perhaps for a different reason than Moreover - that Google is too difficult to take head on.
I may sound arrogant, but I cant help feeling that I've seen this train wreck before, and am also a particular fan of
Dave Sifry who I think has had a rough deal lately.
Technorati is a Blog Search Engine, period. It has some peripheral features that help differentiate, but they are peripheral. Anyone who thinks differently on the board or within the company, is a liability.
I say all this because Google have finally got their act together with Blog Search, and the window of opportunity for Technorati to be what they are, and what they created, is closing.
Dapper fills a perfect niche.
People forget that before RSS there was screenscraping. And that after RSS there is still screenscraping. Most of Google News is scraped and does not come from RSS.
Amazingly, because nobody really puts any useful metadata in RSS, you still need to screenscrape to produce useful aggregation services.
Other than enterprise companies such as WebMethods which had a scraping tool as part of a web services builder, or the innovative Junglee that was snapped up by Amazon before the last .com boom got underway, nobody has built an online screen scraping tool, despite the fact that its actually a massive gaping hole in fundamental services of the web.
At Moreover.com, RSS was largely useless to us, because you can't build a news search engine without full text, and the bigger news sources don't want to output full text RSS, without prior negotiation. So, like Google News, we were managing tens of thousands of scrapers, for search engines like MSN and Yahoo, - which is a pain in the ass.
Because this is a pain in the ass, Dapper is a damn good idea, but because people imagine that RSS is something its not, people may not realize.
If the right people get to using it, Dapper could become a prime mover in making RSS be what people think it is, allowing people to build good vertical search services such as real estate where you want to search by number of rooms etc.
Waiting around for people to create a real estate module for RSS may not be practical. It would be better to scrape and then make the module yourself, using Dapper.
For Dapper to succeed I'd guess that they need to focus on a community of content aggregators, rather than be purely a software service.
I just noticed that Google add automatic Map Links when something that looks like an address appears within a message in Gmail.
This kind of on-the-fly detection of metadata to create searches could be used for auto-dialing phone numbers or adding appointments to a calendar - but I guess we'll have to wait for a Google Calendar product for that.
"Gmail makes it easy for you to keep track of your packages, and map out directions to your destinations; when you open a message that lists an address or package tracking number, Gmail shows you handy links to maps and directions, or your package's delivery status."
Although unannounced publicly, Google appears to be testing its Yellow Pages killer, maps based advertising.
If you do a search for Hotels in New York on Google Local, you get something that you don't get for a search for 'hotels in San Francisco' - ads. Right there as little blue map balloons rather the red, algorithmic, local search results.
Not only are the ads local, but they are contextual i.e. hotel searches bring up sponsored results for local hotels.
In some ways this is a relatively obvious move, however its big news considering that:
1. The Yellow Pages advertising market is bigger than the entire existing online search advertising market.
2. Offline Yellow Pages directories will clearly be replaced, over time, by online products, and it looks like maps are how this plays out.
3. Ad products are where Google makes the money that justifies its gargantuan Market Cap. so a new ad product is a big deal. Now, alongside Adwords and Adsense it has a third revenue source that is in a bigger marketplace.
With ads - Google Local - hotels loc: New York, NY
Without ads - Google Local - hotels loc: San Francisco, CA
Buried within the comments of Jermey Zawadny's post about Feedster is this comment:
"I don't recall Feedster ever being all that useful. But I also don't find Technorati particularly useful. Why can't someone just create a simple search engine for feeds/blogs?"
The truth is that it is very difficult to build a search engine with real-time updates, since search engines are optimized for retrieval and usually use batch indexing. In addition, the majority of weblogs are spam, further compounding the problem.
Blog search, which may once have seemed niche, will eventually be a standard part of search engines. At the moment, nobody, including Google, have a weblog search product that works.
If they did it would be very useful.
The real reason this is important is that it has nothing to do with weblogs, long term. There are only two things that matter in search - freshness and relevancy.
At the moment search engines like Google do not have a button that says order results by date - they will, eventually, and from that comes blog search or from blog search comes that.
Feedster Will Die in 2006 (by Jeremy Zawodny)
This is my personal opinion and does not reflect any company policy.
Most web content is published and then indexed when a search engine finds it, taking up to 30 days. In the past submitting your site to a search engine was the done thing - now its coming back, only better.
Search engines have completely different indexes for news and weblog search, because the indexes need to be updated more quickly, to be able to do this they cannot search the entire web every few minutes but need to be alerted - or pinged. Currently, 'pings' to sites like weblogs.com or ping-o-matic or blo.gs say that SOMETHING has been updated on a weblog or news site. Specs such as RSSPing change this to a ping that says WHAT has been updated. If all pages being published on the web did this (and there is no technical reason why they couldn't), then search engines would not need to crawl websites and search engines would be updated instantly.
Search engines are measured on how much, how relevant and how fresh. Pings are the answer to the fresh bit.
Mike Graves points out that Verisign plan to build value add services on top of pings, but acknowledges that pings themselves should be free:
"Ping services are not a profitable business, in and of themselves. Pings are free by tradition and by necessity. Attempts to introduce cost or latency into the ping layer would be self-defeating; the network simply routes around such problems. A free, open, scalable service fabric for pings is a powerful base for us to build value-added services on, however."
This is good news because a single vendor 'owning' pings would really mess things up for publishers and Internet users in the long term.
In order to maintain innovation and development in weblog style publishing, RSS syndication and possibly even, in the long run, search, publishers such as SixApart should now bake the default ping to a (soon to be) non-profit service such as Ping-O-Matic (unless Feedmesh gets its act together) who would then pass it on to weblogs.com, blo.gs etc.
Alternatively, Verisign could keep Weblogs.com in a non-profit entity and develop premium services within Verisgn itself. This was pretty much how Dave Winer had things, separating church and state between his own publishing engine and Weblogs.com, so people trusted him to keep it neutral. The benefit with this option would be that there needs to be money from somewhere to make pings reliable and filter out the spam. The amount of money or infrastructure needed is not that great. I would argue that despamming, if it is by authentication, isn't part of the value add but that custom subscriptions to alerts on topics are, but that's debatable. In addition, despamming pings doesn't need heavyweight authentication like certificates because the publisher to ping server ratio is not many to many. The problem with a Verisgn controlled root ping server (even non-profit) is that there are other large companies with ping server aspirations, such as Yahoo, who own blo.gs. There may need to be a truly neutral ping service for there to be a central one.
If this does not happen, 'pinging' will disappear as it either: Balkanizes, with companies who have both publishing and search products such as Google or Yahoo refusing to ping Verisign, or each other; stagnates with a single vendor having a lock on the whole thing, stopping competition and therefore, evolution.
The Internet works because nobody owns the roads. Keeping the infrastructure free and making money at the edges is what preserves the marketplace.
Welcome to the Infrablog: Weblogs 2.0
Whatever Web 2.0 really is, and in some ways its an empty 'container meme' for a meme that will morph into whatever is most convenient and successful, Yahoo are looking pretty well equiped to give Google a run for their money in the more media centric worlds of social applications and publishing. When did you last use Orkut? When did you last use Flickr?
With a media savvy exec. team and some small but smart acquisitions: Oddpost; Flickr and now Upcoming, Yahoo have the people, the components and the technical approach to create a synergy of social applications with next generation UI.
It used to be that using online apps. was a trade-off of functionality and performance vs not having to worry about maintenance, upgrades or backups or ability to move from one machine to another. With Gmail or Oddpost, there is no trade-off, my desktop email client crashed when I had a GB of email and took minutes to search for anything. Web email is now better all round than desktop versions.
As Anil Dash pointed out, the genealogy of AJAX traces back to the Oddpost founders' previous project, the Blox online spreadsheet application.
I wouldn't be surprized if five years from now as the advantages of AJAX style UI, trivially simple RSS driven web services and Weblog driven approaches to online publishing nibble away at Microsoft Office and end up coming full circle with a Blox like replacement for Excel.
Google Maps and Gmail show that their in-house development is strong, but the innovation happens at the edges and with billions in the bank and a competitor that has done some smart acquisitions lately, Google could perhaps do with spending some of that cash on acquiring.
The spread between Google and Yahoo's shareprice/earnings multiple may narrow.
Yahoo Inc. Acquires Upcoming.org
Google is to build a gigantic campus in The Silicone Valley
'Silicone' Valley - that would be the San Fernando Valley where all the porn stars hang out, I guess.
Number of households in US: 101 million.
Local Services market value: $600 billion annual.
Household services: $180 billion annual.
Amount spent on local offline advertising by contracting and real estate businesses: $25 billion annual.
Dotcom investment in 10 online services during boom: $250 million [they were keen but too early]
Amount spent on advertising local services to households: anywhere between $50 -$90 billion annual [this is the biggest untapped revenue opportunity for search]
Largest category of services posting on Craigslist (taken by looking at a sample 2 days of postings): Sex services, 40% [i.e. Craiglist not a player yet, outside of jobs and real estate]
Largest category of Yellow pages advertiser: Attorneys, $856 million in 2001.
Largest single event resulting in Yellow Pages use: eldest daughter gets married [personalized search and user profiles will be important]
Number of Overture searches that explicitly have a city in the search: 4%
Number of Overture searches that have a term such as 'lawyer' or 'doctor': 8-11%
Years that Dex (the largest online version of offline Yellow Pages) have had search: 1 [the existing Yellow Pages providers are incumbents]
US Yellow pages advertising revenue: $13 billion annual
Local advertising budget in US: $22 billion
Total size of paid search marketplace: < $10 billion annual [i.e. less than existing Yellow Pages market]
This is why the battleground for search will be over Yellow Pages. It is now clear what the product will look like, and maps will play a big part.
Google ups ante in mapping rivalry | CNET News.com
Tom Foremski on what would happen if someone created a treasure hunt with a large cash prize awarded to a single click of an unpublicised adwords ad. A subsequent clicking frenzy could drain advertisers' accounts, prompting them to ask for a refund.
This hypothetical idea is part of a more serious problem - pay-per-perfomance advertising is open to fraud - when you click something, money drains out of an account - this doesn't happen with TV, print or radio ads. See Adbombing.
In the same way that companies like Paypal spent a considerable proportion of their resources dealing with fraud, so will Google.
If Google succeeds then its anti-fraud measures will be a competitive edge. If it fails there will be a problem.
The moral to all this is that Google's business model landed on their laps via Scott Banister at Idealab, it is a new model and its weaknesses are not yet exposed, let alone tested.
Given the risk (and the fact that the existing behemoths like Ebay are beginning to plateau earlier than thought) a price/earnings ratio of half its current level of 140 would seem optimistic.
Google News has been in 'Beta' for nearly two and a half years.
Several million beta testers a day for nearly a thousand days, i.e. more than a billion (non unique) beta testers.
That's quite some QA.
I wonder if this is a record?
Google launches news service - Computerworld
News search has one feature that is still lacking in search engines: sort by date. This is something which will eventually be a core requirement, and the search engines seem to be asleep at the wheel.
Most search engines sort by relevance, but for subjects which change rapidly such as technology, freshness is an important component of relevance.
Having just searched for some software on Google I realized that the top results were 4 years old and useless.
It is not technically difficult to create date ordering, but it is computationally expensive and requires comparisons as documents are crawled. There is, however, one area where searching by date is already there: weblog searching.
The model where content sites ping a server when there are updates soves the date problem cheaply and increases overall relevancy. The ping model is as different from the way search engines currently work, as push is to pull.
This is another compelling reason why the search engines will be caught off guard if they do not pay attention to weblog style publishing and ping servers, not just as an additional feature, but a component that is core to what they do.
It sounds like a very magnanimous thing for Google to do - to build a virtual library of Alexandria, but there is a solid business reason as well.
One of the simplest ways to game Google is to scan out of copyright books, rare ones ideally, boost Pagerank by buying hard links, and serve Adsense against the results. This is commonly done currently, with specialist Dictionaries.
However, Pagerank only really works if you have original content, i.e. stuff that is not already on the web; slapping up a copy of the works of Shakespeare won't do. If Google scans out of copyright books, and serves them up itself, then attempts to trick Google into handing out Adsense revenue without generating any content will not work.
Google adds major libraries to its database
With desktop search Google now has an application that makes it much more likely that you will continue to use their search engine.
They have created a switching cost - after spending several hours indexing your drive, you are less likely to switch to a different service.
Although there is a lot of hoo hah about desktop search, its still amazing that it took till 2004 for searching your own machine to become a mainstream app, when you have been able to search thousands of other computers around the world, within an instant, since the last millennium.
Expect Microsoft to counter aggressively, their business is built around owning the command line or desktop and they will likely build in indexing out of the box, meaning that Google desktop users will end up with two or more indexes.
Whatever Microsoft do, Google have shown the way forward, their desktop search makes your desktop just one more search tab. It brings your desktop to the web rather than the web to the desktop and this seems like a much more logical UI experience.
Google's attempt at an auction could break a piece of the cronyism that has plagued corporate America and has caused huge failures from the demise of Enron to the collapse of the technology bubble. Middle men creaming large fees for little value-add and dolling them out to friends is not a good thing regardless of whether you are a free market evangelist or not. This is why I am so surprised that people like Dan Gillmor are choosing to attack Google's offering. Google's offer price is an attempt to derive price from real demand, not what generates profits for middle men. This benefits small investors in the long term.
Why aren't Google's PR team on the offensive over this!
"If Google's offering works...then this IPO would legitimize an alternative to the traditional IPO that will diminish the power of Wall Street investment banks. Other companies, companies with lower profiles than Google, will have a new alternative for raising money. Wall Street doesn't even like to think about that possibility."
"If, on the other hand, Google's IPO fails -- if not enough investors bid, or if the price is too low, or if the IPO sinks, leaving hordes of angry individual investors and the company with egg on its face -- then the auction model will go back on the shelf and Wall Street investment banking will go back to business as usual."
Via Anil
MSN launches Newsbot, Microsoft's answer to Google news.
If you look at the URLs as you mouse over them - its all powered by Moreover..
Google has recently introduced banner ads for their adsense program, so perhaps all the talk about how text ads perform better is not true.
You could argue that banner ads are a different thing, that they are richer media and thus brand advertising. But the difference is that Google's image ads do not pay publishers on an impressions basis, but for clicks, just like current contextual text ads.
Adwords works. People click on text ads which are relevant to a search because they are actively looking for something. But perhaps when ads are served alongside static content, the conversion rate is lower, and perhaps people are starting to ignore them now that the novelty has worn off. So maybe in your face image ads are back, for good reason.
The problem for Google is that image ads are not only 'in your face', they are very much in the face of a website. With the renewed confidence that content publishers now have, my guess is that CPM impressions based advertising will be back. If publishers have leverage they will want the guaranteed revenue that being paid for traffic gives them.
Google got rid of all CPM advertising making a big bet on the best of both worlds CPC model that sits in between an advertiser and publishers needs. If CPM is back is that bad news for Google?
Google's ad plans provoke grumbling
HotBot's New Desktop Search Toolbar:
"HotBot's new Desktop Search utility not only searches the web, it indexes files and email on your computer, making them searchable as well."
It's only a matter of time before Google do this and perhaps what has held them back was fear of being too aggressive against Microsoft.
As I've said before, it is crazy that we can search millions of other computers, thousands of miles apart, more quickly than our own machine.
Microsoft should have done this 8 years ago, we should not have to wait because of the stagnation created by a monopoly.
Another meme spreads round the web, the Sergey pic is on Hot or Not:
Is Sergey HOT or NOT?
My enemies' enemies are my friends, not when my enemy's enemies are hosted by my enemy.
Sign up for the Google Newsletter at Yahoo Groups
Another classic via NTK
Search Beyond Google outlines newer search engines' challenges to Google and they all seem to be based upon different approaches from Pagerank.
Was Pagerank what really made Google or was it the fact that they had elegant usability and design, weren't a 'big bad company' and were perceived to be cool for the people that matter when building a tech company - the techies, the early adopters who are the biggest users and potential evangelists?
Pagerank is arguably obsolete at this point, as weblogs and trackback and Tripadvisor.com demonstrate.
Surely the trophy collection of PhDs that work at Google are better used to optimize Adwords or tackle the physics of cooling fans in the server farms than all be tweaking Pagerank?
Is search a software problem or is Google an advertising company with great hardware hosting skills, that pretends to be focused on the arcana of search algorithms because it sounds cool if your PR needs to focus on what's hip in the tech world?
"the rumors are flying again about Jeeves being in play. CBS Marketwatch is fueling them, saying AOL might buy the company and drop Google. I don't think so, but you never know"
Certainly if Ask Jeeves don't get bought they look very lonely out there as a destination site powered by Google ads revenue. They are less vulnerable than Looksmart in that there is little incentive for Google to ditch them as a channel in the same way that MSN ditched Looksmart as a provider (although Google have leverage in terms of split on ad revenue).
I agree with John that they won't get bought by AOL. They have traffic and a search product but their ad network is provided by Google. AOL have more traffic and search, despite the hype, is a commodity. AOL would be better off with an ad network.
At $40 billion a year for traditional print advertising (10 times the size of the existing search engine advertising market), the online Yellow Pages advertising market is the biggest revenue opportunity for search engines now that the pay-per-performance revenue model is cemented. Since most services transactions happen offline, Google/Overture style PPC (Pay Per Click) is the perfect way to charge advertisers.
Latest figures show that search for local services is twice usual estimates, at 25% of all commercial searches online.
The market for local services has traditionally been owned by the phone companies who used phone numbers as the key for Yellow Pages listings, however, Yellow Pages publishers have been lazy and arrogant and are just realizing that they could be crushed by the likes of Google, since the majority of revenues will soon come from online advertising. Traditional publishers' online Yellow Pages are usually very poorly executed, for example, Dex only introduced searchable listings (instead of browsing a directory) at the beginning of this year.
The model for online local services is probably Citysearch who offer paid listings alongside the directory and include value added content such as ratings. Craigslist should be in there but the lack of focus means that the biggest services advertised have gravitated to the seedy or low cost end of the market, Craiglist's biggest services advertisers seem to be prostitutes, whereas Yellow Pages' are doctors, dentists and lawyers.
Although Yahoo has had a local services tab for a while, it hasn't linked this to pay-per-performance advertising, however, an Overture service is in the works. In addition, Google will surely ditch their increasingly defunct DMOZ powered 'directory' tab for the Yellow Pages service at Google labs. For Google, Yellow pages may have an added bonus - since local services require a search term plus some information about yourself, such as location, storing this information in a profile benefits users and helps build in a 'switching cost' to lock in users. Perhaps this is the kind of profiling service they should morph Orkut into.
There may be an opportunity for weblog publishing tools to package a product geared to small business websites. In order for PPC local services ads to work, they must click through to a webpage about the service, but most of the 100M local businesses in the US do not yet have a website.
TiVo and PVR wishlist:
1. All PVR's are sold as commodity hardware with no signup or subscription fee or tie in to Cable or Satellite.
2. TV listings are provided for free by ad supported online services.
3. Opt-in targeted ads based upon your personal and viewing profile (with ability to remove items from viewing profile) are served alongside listings.
4. You have total control and ownership of your profile to block certain advertisers or limit your profile.
5. Either onscreen or via a webpage you can buy from ads based on what you have watched. For example if you watched a travel show about Hawaii you can choose special vacation deals from your nearest airport (you can store your zipcode in the profile), if you want to buy what they are wearing on Sex in the City (heaven forbid), then you can.
Basically I want to walk into a store buy a PVR with no signup and use a different company for free listings that serve the equivalent of search engine text ads but for TV.
Perhaps Google should think 'inside the box'.
Three major players in search, Yahoo, Google and soon Microsoft makes for more players than some of the other big Internet services, occupied by the likes of Ebay, Amazon, Netflix.
Here's a wild prediction for the first massive merger of the rebound: Microsoft will buy Yahoo (if they can get away with being that aggressive).
MarketingWonk, on agency snobbery towards adwords/adsense:
"I'm getting negative vibes from the professional ad community, but mostly I think because syndicated text ads isn't glamourous [sic] and it cuts out a lot of middle men and agencies."
John Battelle wonders if you are compelled to stare at the Bruce Clay search engine landscape chart via Kottke.
If you do, you will notice that its not quite what it appears. Consolidation means that there are only 8 companies involved (I've highlighted them in separate colors above) and the outbound link from Looksmart has now gone, leaving only 2 3rd party providers, Google and Yahoo.
Google AdSense: A better way to make ad revenue
Hmm.
Update, this is even more absurd:
Perhaps the strapline should have read: "Google Adwords, Ads that work unless you are in advertising and are an expert in ads in which case adwords are far too subtle" or "Google Adwords, ads that don't always work" or "Google Adwords, as not used by Google and not suitable for advertising execs or children under 5"
For the sake of brand marketing instead of 'marketing marketing solutions to marketing people' - as they used to say in 1999, Google should eat its own dogfood and advertising people should feed from the same bowl.
"By submitting, posting or displaying any Materials on or through the orkut.com service, you automatically grant to us a worldwide, non-exclusive, sublicenseable, transferable, royalty-free, perpetual, irrevocable right to copy, distribute, create derivative works of, publicly perform and display such Materials."
OK- this is not unusual, even if aggressively worded, but considering that Orkut has a great deal more personal profiling than most social network tools, isn't it about time that people started an identity system where people actually owned their own identity.
Think how useful it is for Google to have your personal profile in order to target ads at you, particularly as they go after the $25billion yellow pages advertising market. People are giving valuable information away for free as part of a game.
Search for 'Linux Windows' on MSN and Google respectively:
MSN Search: Results 1-15 of about 18 containing "linux windows"
Google Search: linux windows, results 1 - 10 of about 8,990,000
A slight discrepancy in the numbers.
Update: see comments, MSN does in fact return lots of resuls if you view the second page.
Microsoft is working on search integrated into the OS:
"The tools could also permit Microsoft to undermine the utility of commercial search engines such as Google by making its own software the easiest place to initiate an investigation. Spell-checkers, after all, were once independent applications too."
This is no surprise, building search into the desktop is something that Microsoft will allways have an advantage with. But it does raise an important issue: given that documents on your hard drive contain personal and sometimes confidential data, it would be alarming to see ads based upon the contents of these documents served alongside searches results on your hard drive.
Without an ad based revenue model for desktop search, Microsoft would have to either make web search separate with a Google competitor via MSN or subsidise a hybrid web/desktop search as part of the OS. Google would be up against a free product and this is all too reminiscent of Netscape vs. IE.
The silver lining is that Google's advertisers pay not searchers and Google ads, clearly marked as such, are a benefit to users which actually enhances the service.
Clay Shirky is continuing to set himself up as the anti-semantic web guy. Its an easy target and good for spin. But, after all, what is anti-semantic if it isn't meaningless.
Clay on the Yahoo ontology: "it sucked. Sucked sucked sucked. We didn't even know how bad it sucked until Google came along and (its hard to remember this even five years later) saved the Web from drowning in its own waste."
Well, three things:
Google is a search engine, and does pretty much what Altavista did 5 years ago, before they stopped being just a search engine. They sensibly ignore meta tags, but that was largely to do with people deliberately entering false information. Yahoo's category search (Yet Another Hierarchical Officious Oracle) is becoming a search engine 'Yase' because its difficult to impose ontologies on the web as a whole.
Things that aren't really search engines, like Amazon and Ebay, or the classification of species for that matter, would be useless without some kind of ontology.
Even without a complete hierarchical system for classification, metadata is useful where pure full text search fails. Try searching for a cheap flight on Google, you can't - unless they scrape the metadata.
Many companies use the Google Trademark Complaint Procedure to stop competitors bidding on a company name or product. As this becomes more common practice it may hurt Google's revenue potential and therefore, valuation.
I wonder whether this leads to the possibility of people trademarking terms and expressions specifically so that they can use them within Google ads?
This seems to raise a general issue. Given that a trademark is awarded within a specific industry, but a search term is not aware of the context e.g. a search for 'Windows' could be for building materials or for software. What is to stop people registering trademarks outside of their common use in order to block keyword searches?
In other words, given that 'Windows' is trademarked, why couldn't you trademark 'blog' as a type of candy and block anyone advertising against it regardless of the context.
Microsoft owns the desktop. At one time it owned the command line, the C:\ prompt, and now it must own the command line that connects away from the desktop to the Internet.
This is the core of what Microsoft is about, its unstated mission statement, and this was why Microsoft had to react quickly to the threat posed by Netscape.
Google owns the command line to the Internet and Microsoft cannot afford to concede that to them. That is why they may indeed have explored buying Google. Even if the reports of this are not true, as is probably the case, the rumor itself signals a warning shot that Google are on Microsoft's turf and so perhaps lowers the price that they could buy them for post IPO.
Google is set to battle two giants, Microsoft and Yahoo. Google have the brand, Microsoft the ability to put search directly into the computers that people buy and Yahoo have the portal extras that are very difficult to introduce without losing focus.
Google, who has done everything right so far will do everything it can to avoid becoming a Netscape to Microsoft and an Altavista to Yahoo. It will need to lock in customers by making it difficult to switch rather than relying on brand alone.
One of the benefits of 'CPC' based online advertising, where you pay for clicks rather than impressions, is that if the advertiser works out the average number of clicks that result in a sale she knows how much a click is worth for it to be profitable.
Tracking this kind of 'conversion' used to require custom code, or use of a third party service such as GoToast, but now that Google have built it into Adwords, things should be a lot simpler.
In theory, if you have an ecommerce site you should be able to setup Adwords so that you are guaranteed not to lose money.
Google Adwords: Conversion Tracking FAQ
You may be in breach of Google’s new Adsense terms and conditions – but their principal revenue model may also contravene your content license.
Matt Haughey looks at how to make money form a weblog by focusing on one thing and using Google's Adsense.
But Google itself makes money from text ads based upon this content. Following Matt's argument, the best model for a weblog search engine or aggregator is Adsense - aggregate weblogs into specific focused categories or search returns and run Adsense along the side. If more focused content drives more Adsense revenue, then let the aggregator create that focus through topics.
But there is a problem with this. If you aggregate from sites that are funded by Adsense then you are, in effect, ripping them off by making revenue from other people's content and denying them their principal revenue in return. Tough luck perhaps, but in actual fact many weblogs including Matt's PVR blog explicitly prohibit using the content for commercial purposes, via a suitable Creative Commons license.
If your site is funded by Adsense and has a 'no commercial use' creative commons license, then any search engine or aggregator that in turn uses text ads based upon search returns or aggregated topics, is making money out of your content in exactly the same way that you do, but with less effort.
One solution to this would be a revenue share on ads served against your content (and this requires ads at the individual post level). This isn't going to happen on standard search engines, however weblogs have one valuable asset that gives leverage - ping server notification.
Ping servers allow real-time search of weblogs unlike the 15 minutes or older returns for most search engines. Perhaps bloggers should only allow access to ping information, if there is revenue split on text ads served against their content?
Revenue-split advertising models may sound too complicated to play out in the real world, but if content providers and search engines both make their money out of contextual ads, or content providers allow repurposing and syndication by serving up full-text RSS content, without branding or ads, then this will be inevitable, in order to avoid the circular mess of when both parties serve up ads e.g. in RSS feeds themseves as well as in searches accross RSS feeds.
I predict that within two years most business weblogs will be based on 'Adsense-like' advertising revenue and that weblog aggregators and search engines that offer searches accross content as soon as it is posted will split revenues on ad clicks between the publisher and the aggregator.
Most people are familiar with searching a database and most people are also familiar with searching a search engine, but because the latter was not always the case before the web, these two products are usually so different at their core that combining the two either makes searches slow or highly structured searches impossible.
Metadata is difficult to infer, but if it is inserted at the point of publishing, as weblog style tools potentially allow, then to search the web like a database will require a product like Cerisent's. There are other similar products from people like Software AG, but Cerisent's is the only one where the core is written at a low enough level to scale to the requirements of a hybrid database/search engine.
So it seems that as Yahoo has acquired or de facto acquired Inktomi, AltaVista, FAST and Overture in order to compete against Google, Microsoft is building search in-house.
Google has the brand and focus, Yahoo has the community and customers for value-add extras and Microsoft have the ability to easily combine web and desktop/intranet full-text search. But where have AOL been in all this? AOL is looking increasingly vulnerable with no visible plan for search and a dial-up service that looks expensive when compared to broadband.
Andrew Goodman deconstructs the latest search engine stats:
Google Sites 32%
Yahoo! Sites 25%
AOL Time Warner 19%
MSN-Microsoft Sites 15%
Ask Jeeves 3%
"Since Google powers both Yahoo Search and AOL Search, if you assign the lion's share of searches on those portal properties to Google, you arrive at the conclusion that Google might be powering 60-70% of all online searches."
The searches numbers are almost irrelevant, however - Google won't have a deal with Yahoo forever, and it won't be getting any ad derived cash, so the paid search (Adwords etc.) numbers are more interesting:
Google Network 54%
Overture Network 45%
As Andrew points out, are Findwhat and Looksmart really just a rounding error blip? Something weird here.
Traffick | Minding the Search Engines' Business
I had figured that "Overture get bought by Yahoo or Microsoft within a year."
So after the consolidation amongst the algorithmic search the paid for search space is also coagulating around the principal search destinations.
The major hubs of the web will now be Yahoo, Google and MSN, but MSN seem to be playing a waiting game, perhaps they are arming internally, ready for a battle royal.
Microsoft is flexing its muscles with new in-house search technology for MSN. As the new big three search companies Yahoo, Google and Microsoft really engage they will have to compete on the paid for search as well as algorithmic search.
Today Findwhat bought Espotting for $163 million. What's the bet that Findwhat don't get bought by someone further up the food chain like Overture, and that Overture get bought by Yahoo or Microsoft within a year.
Mercury News | 06/18/2003 | FindWhat.com will buy Espotting for $163 million
One of the differences between Overture and Google is that whereas Overture rank keyword based ads on the amount bid, Google rank them according to a function of this plus clickthrough/impressions history.
In other words, does Google have a clickrank algorithm that is analogous to pagerank?
How could you have made more than 10 times your money in less than a year, subsequent to the .com crash? - by buying stock in a .com
Last July Ask Jeeves teamed up with Google to have Google supply advertising alongside search. The deal replaced Overture and involves a revenue split between Google and Ask Jeeves, with the majority going to Ask Jeeves.
Since most of Ask Jeeves' revenue comes from this, and since most of Google's revenue comes from similar, this is a vicarious way of tracking Google's performance while they are still a private company.
The results are outstanding, Ask Jeeves is now worth 13 times what it was before the deal with Google.
Contra Costa Times | 07/19/2002 | Ask Jeeves, Google team up for profits
Esther Dyson (who is nobody's fool) asks why Google have bought Applied Semantics. Craig Silverstein replies that it has to do with advertising, so Esther asks if it has anything to do with the 'Application of Semantics'. Craig has nothing to add - and a murmur sweeps the room as if to say, ah they are up to something with the Semantic Web... crap.
1. Applied Semantics and fancy fuzzy search or categorization has nothing to do with the Semantic Web. If anything the polar opposite type of search technology is required - something that takes advantage of semi-structured documents such as an XML or RDF database.
2. Google is an advertising company. Search is a fairly finite field in computing - and Google's research team is now bigger than any University's - they have this area sewn up. What they don't have sewn up is the technology and services required for advertising, and this is how Google makes its money.
The problem with this meme not propagating is twofold:
1. An advertising company doesn't sound cool - so people prefer to think of Google as a cutting edge technology company (which it obviously is as well even if that role is coincidentally more useful from a PR perspective).
2. People love conspiracies or hidden agendas - ooh what was the agenda behind the Blogger acquisition - probably none other than access to a big user base that puts metadata around links and therefore can help keep pagerank on the rails.
The really interesting thing about the Applied Semantics acquisition is the lack of a cool, secret, Semantic Web skunk works being built by a team of crack engineers at Google - the really interesting thing is its banality, they are buying bits and pieces to help with targeted advertising, and it may be that all they needed was some 'intellectual property' or patents, so that they could develop a refined ad delivery system without infringement. Nobody likes a boring story, however.
Good analysis of Applied Semantics by Azeem
The article below claims that Google has invested in the 'Semantic Web' with the acquisition of Applied Semantics. The only connection I can see is the word Semantic in the name of the company.
What this does seem to show is that Google is building up its armory of weapons to deal with analyzing content to produce better targeted advertising and that its core relevance ranking software isn't enough.
Google Invests in the Semantic Web - search engine news blog
Anil suggests using a RAM disk as a solution to Windows' shabby 'find'.
Well yes and no - sure you want to have your search index in RAM, accessing RAM is 1000 times faster than disk access, but the main problem with Windows is the lack of a decent searchable index in the first place.
The Longhorn version of windows will have full text search built properly into the filesystem. By the time that Microsoft release this, XML databases, or even better, graph based databases may make their efforts appear to be already out of date. A filesystem index is a perfect application for a native XML database.
"Google argues that SafeSearch is designed to err on the side of caution."
Report criticizes Google's porn filters | CNET News.com
Er... so what? Imagine if it didn't err on the side of caution. Any system such as this is either optimized for relevance or retrieval, and Google's is optimized for relevance.
The best solution to filtering is an editor/software hybrid. Editors cost money, so the only practical way of dealing with this is to have site owners become incentivized editors and submit to a growing list of sites which wish to be removed from a software determined blacklist.
Interesting article on content based advertising.
Gil Elbaz of applied Semantics claims that "the very fact that search engine algorithms remain largely keyword-based means that they aren't particularly sophisticated in learning what a page is "about." I think this is optimistic.
Google have the expertise to develop a concept based approach, possibly using intellectual property gained through their acquisition of Outride, but they certainly need to get their act together here or text ads advertising Hummers from suvssuck.com aren't going to be that impressive.
This is particularly important when you consider the following: "While clickthrough rates might indeed be lower, Google claims that their tests show that post-click behavior (conversions to sales) resulting from content-targeted ads is similar to that seen with search engine advertising."
One good thing to come out of consolidation in the search space is the general realization that Google is, first and foremost, an advertising company with an excellent brand.
It is another advertising company, Overture, that has bought some of the main search engines, not the other way around.
Search technology is a commodity and subtleties like PageRank are icing on a cake that others have the recipe for. The main issue with search from a technical perspective is scalability and, as FAST has shown, Google is not the only one to have figured this out.
Info on Longhorn, Microsoft's planned file system:
Longhorn will finally allow you to search a single Windows machine with similar ease to the way you can already simultaneously search hundreds of thousands of other computers, via a web search.
But, as Danny Ayers points out on RSS-dev, Longhorn is based on a relational file system, something which is perhaps obsolete for the purpose. An XML or even better, a graph based model would be more suitable.
"Perhaps a little unimaginative of them to use a relational store (a graph model would be a better match for the networked computer environment IMHO), but I suppose they've already got the code. I guess at least it will mean that "Find File" will take days rather than months..."
This is huge- ok, perhaps overstating, but remember, search ads are a 1.5 billion dollar business and the following extends Google in the general ad space alongside companies like DoubleClick. The first signs of what the Google/Blogger combination will yield are shown with Google's latest Adwords initiative.
Google has just launched adverts which are based upon the content of a web page, for an example see the old Industry Standard website.
This is a major new direction for Google since it extends their main revenue source from adverts on their own and affiliated search sites to any website. It also marks a trend away from the banner ad - in fact web advertising will increasingly consist of targeted text advertising and rich media adverts.
The key to the targeted ads is that they change according to the content of the page (Google continually spiders the pages and serves different ads based upon what it has spidered and indexed), in other words they are perfect for sites with dynamic content - and this is where weblogs come in.
If you look at the adverts on Blogspot you will see that as of now, they are also based upon the content of the page. Because Google owns Blogger, they can index the pages in real time and thus target ads immediately there is a new posting.
Three things are missing to make this a killer app:
1. The ads should be targeted at the level of the posting as opposed to the page, then they could be served inline even when syndicated via RSS aggregators.
2. As Dave Winer has suggested, Google should look at pings to weblogs.com to allow real-time targeted ads for users of other tools such as Userland and Moveable Type.
3. The Google indexing engine needs to detect tone in the context of a keyword mention. For example neither the advertiser nor the Blogger would want adverts for SUV's appearing on an environmental site that has a posting against gas guzzling cars.
So I was wrong last week, but correct originally. Overture have indeed bought FAST.
Despite the drop in their share price, this is a good move by Overture. Google has not yet raised cash from an IPO and so can only offer stock. Overture is mopping up the loose ends in the search landscape by offering cash and is covering itself against longer term weaknesses.
Overture to buy search services | CNET News.com
With the majority of their revenue coming from two partners who have destination sites and therefore can beat Overture down on revenue split (or 'cost of traffic' as Overture call it in SEC filings), Overture needs a destination site, but Google is way ahead of Altavista in terms of users.
This consolidation is bad news for FAST, Microsoft could possibly buy them for MSN, but if inclined to buy at all they may look at AskJeeves.
So what does the search space look like now:
Google
Syndicated and destination site search, paid listings (adwords) and relevance ranked search.
Yahoo
Syndicated (Inktomi) and destination site search, paid listings (they are probably thinking of building their own) and relevance ranked search.
MSN
Destination site search
Overture
Syndicated and destination site search (Altavista),
paid listings and relevance ranked search.
Possible acquisition targets:
AskJeeves (Microsoft); FAST (Microsoft); Espotting (Overture or Google)
The two most important things on the web are publishing and searching.
Three and a half years ago when I stepped off the plane at SFO with my carpetbag, I had two meetings lined up, one with Autonomy and one with Evan Williams from Pyra, a search behemoth and a tiny publishing (actually collaboration software) startup. Pyra was more exciting.
Evan and Meg Hourihan had developed a product that was designed for people who needed to update websites often and easily. To do this they had to produce a beautifully elegant piece of software. Blogger along with Manila seemed to point to something profound, what happens when the web becomes two-way.
Weblog publishing tools allow you to create embedded meaning within documents and they embed that meaning where it is relevant - in the nugget of information that is published as opposed to a web page. RSS syndication and the virtuous circle of searching syndicated content, publishing commentary and re-syndicating content, allow for the beginnings of a whole new way of interacting with the web. They are the foundation stones of the mythical semantic web.
So will an acquisition allow Blogger to flourish? There are a myriad of opportunities for a symbiotic relationship between search and publishing, but Google is now a medium sized company that is focused on ad revenue not cool web applications. For all that, at the very least, realtime search of weblogs will complement news search and Google had the sense to do something here. I just hope they don't produce a weblog product called Booger.
Well done Pyra - and well done Google.
Searching for people is one of the most difficult problems for search engines.
"Surnames cause the biggest problems in genealogy... In one document, my ancestor's surname appeared as Mury, Murry and Murray. Samuel's surname is Murry on his headstone. On the stone of his son Levi, it's Murray."
The most widely used standard for searching for name variants is Soundex, however this does not take account of non Anglo-Saxon sounding names and does not work with a thesaurus of known variants.
NameX, which was developed for use on Originsearch was designed specifically for looking at the name variants used historically, as well as phonetic variants. For example, it picks up 56 variants of Mury. It would be interesting to take the Namex concept further and allow you to choose name variants based on the ethnicity or language of the search.
Floridian: How well do you know your name?
Overture's Achilles heel is not owning a destination site. As CNET reports, Yahoo's poaching of an exec from Overture is significant. The barrier to entry is too low for Yahoo to not build its own paid search, avoid a 35% commission fee and sever links with Overture.
Yahoo is putting the pieces into place to replace partnerships with Google and Overture and provide their services in house.
For Google this means that Yahoo will compete with them, but Google owns THE destination site for search and provides its own paid listings.
Overture could become the Inktomi of paid search, beholden to those who own the destination sites, the majority of its revenues come from two partners, one of them being Yahoo. It proved the business model for search, its revenues are impressive and its $1.3 billion valuation quite an achievement, but the business model missed one key ingredient, it doesn't really own the customer. Its growth looks stalled and its future grim.
Yahoo hires Overture search exec - Tech News - CNET.com
Expect FAST, the Norwegian company behind Alltheweb, to be acquired by Overture or possibly even Microsoft.
That will leave 3 players in online search, the rest won't matter.
Lets play a game and imagine that we are in charge of Google strategy.
The diagram above is my stab. There are two principal axes, both of which Google has started on, but do not yet appear to form part of a core strategy. In other words, the Google appliance currently looks like a distraction and distinct business compared with the core search business in which the adwords service is the biggest initiative.
Continuing from the idea of Google beating Microsoft to desktop search, the vertical axis proceeds from Internet search to Unix server based intranet search, followed by Windows client desktop search and, lets say, a Symbian cellphone/PDA client. Components of each search, such as desktop email, and cellphone contacts, can be hosted by Google or combined for access from any machine anywhere via a thin client. Value added services such as sync and backup and security are offered.
On the horizontal axis there is a continuation of the web toolbar into more specific information types such as product search, combined with more filetypes to support intranet search etc. and eventually moving into a universal API that allows simple transactional search functionality to process queries like: 'give me the value of my stock portfolio based upon a stock ticker search'.
The revenue streams then become: ad supported web search; software licenses; transaction based web services fees and subscription based remote hosted services.
The search box extends beyond the web to allow searching files that are located on any machine, from any machine and the search becomes not only aware of the content that is being searched, but triggers different results based upon the context.
A universal command line to access anything from anywhere.
David Coursey, "Here's what I think the next Windows will look like", writes that Longhorn, Microsoft's next generation operating system, won't be ready till 2004 and that it will be later rather than sooner.
One of the key components of Longhorn is a long awaited overhaul of the Windows file system that will allow proper full-text searching. File systems built on top of databases are not new, IBMs AS400s have had this for years, so it could be argued that an update to Windows is not technological progress but fixing inelegant software architecture. The Internet has made the use of full-text search familiar and necessary and the lack of this 'must have' feature in Windows is now embarrassing.
If this is not going to be available till 2004 then perhaps there is an opportunity for Google. Imagine a Google toolbar that provided indexing of files on your computer, rather like that provided by Enfish. Google would search your own computer and the Internet seamlessly. Now imagine that optional extras included searchable, secure backups of your email or documents accessible online. Microsoft's dominance, that has spread from control of the command line to control of the desktop, would be challenged as a design flaw at the core of its operating system was exploited.
So what's going to happen in the online search industry? Here are my predictions:
Overture will buy Espotting who have the biggest market share in paid results in Europe.
Paid results and editorial results companies will merge. i.e. Overture will merge with either Inktomi or FAST.
The industry will be dominated by the above merger and Google, but Google will dominate.
Yahoo and Microsoft will use anyone but Google. i.e. Yahoo will look at an alternative to Google for editorial based search results.
Yahoo, MSN, AOL and Google will be the destination sites that matter. Because Google own one of these destination sites they will not be held to ransom by giving up too much revenue share in the same way that Inktomi, FAST and Overture are.
Very neat - An elegant script to highlight terms when someone arrives at your site from a Google search.
Now if everyone did this I wouldn't need to hit the Google cache to find what I'd searched for on a page.
Online search is expected to generate more than $2.5 billion in revenues in 2004. That gives an aggregate Market Cap. for all search companies of around $12 billion. Assume that Google gets 40% of this and Google would have a potential value of $5 billion.
But Google is perhaps different - it will win a battle with Overture for paid listings, and will dominate the search space - and search is the bedrock of the Internet.
Microsoft owned command line access to your own machine and built on top of that. Google owns the command line for access to other machines, a very powerful place to build a company worth much more than $5 billion.