Conclusion

Conclusion

Excite fared the best in comparison with WebCrawler for precision scores, and in the number of unique documents in the first fifty hits of returned sets from the six proper noun phrase searches. While half the time there was no overlap between the sets of relevant documents, WebCrawler only had one unique relevant document in two of the searches, for the rest its first fifty hits either had no relevant documents at all (search 3) or no unique relevant documents. Very little would be added by using WebCrawler to complement any Excite search.
Excite's Intelligent Concept Extraction search engine did not seem to have a much of an effect on the results. Most of the differences in better precision and uniqueness for Excite can be explained by Excite having a very much larger index than WebCrawler. For those instances where both returned the same documents, the documents were ranked, by and large, very similarily. When non-relevant documents that ranked higher than relevant ones in Excite where examined, I found very little evidence that "correlation of concepts" was being used to rank the documents. Excite did no better job of ranking relevant documents than did WebCrawler, especially after the vast difference in index size is taken into account. This seems to support Linda Barlow's assertion that for the ICE, "'intelligent' is an exaggeration (the apparent intelligence comes from the clever use of statistics, not from a sudden advance in artifical intelligence)," (From Monash Information Services: http://www.monash.com/spidap3.html#excite). It seems the main result of the ICE's proprietary statistical algorithms is that they give documents lower relevancy scores, as overlapping documents list rankings were similar, if not the same (three out of five), as that by given by WebCrawler, despite WebCrawler giving consistently higher relevancy scores to every document.
With Excite returning twice to over 100 times as many hits, returning the most unique hits, overlapping very well with most of WebCrawler's few relevant hits, and with Excite and WebCrawler doing a fairly equal job of relevancy ranking, there is little advantage in using WebCrawler over Excite, or in even using WebCrawler to complement any Excite search.