Think up and write down a small number of queries for a web search engine. Make sure that the queries vary in length (i.e., they are not all one word).
english
Description
Assignment One
Q1. Think
up and write down a small number of queries for a web search engine. Make sure
that the queries vary in length (i.e., they are not all one word). Try to
specify exactly what information you are looking for in some of the queries. Run
these queries on two commercial web search engines (such as google and Bing)
and compare the top 10 results for each query by doing relevance judgments.
Write a report that answers at least the following questions: What is the
precision of the results? What is the overlap between the results for the two
search engines? Is one search engine clearly better than the other? If so, by
how much? How do short queries perform compared to long queries?
Q2. Use
the “advanced search” feature of a web search engine to come up with three
examples of searches using the Boolean operators AND, OR, and NOT that work
better than using the same query in the regular search box. Do you think the search
engine is using a strict Boolean model of retrieval for the advanced search?
Q3. Write
a simple single-threaded web crawler. Starting from a single input URL (perhaps
a professor’s web page), the crawler should download a page and then wait at
least five seconds before downloading the next page. Your program should find other
pages to crawl by parsing link tags found in previously crawled documents.
