This is important for a couple reasons. First, it allows you to specify exactly what pages major search engines can crawl. Second, including a robots.txt makes tracking search engines easier because they always download the file before navigating your client's site. This characteristic differentiates the SEs from normal human visitors.