Master of Science
Soon-Ok Park, Ph.D.
(Clare) Xueqing Tang, Ph.D.
Neng-Shin Chen, M.S.
A web crawler is a piece of code that travels the Internet and collects data from various web pages, also known as web scraping. Some web crawlers are autonomous and require no instructions once started. This project will focus on a user driven web crawler where user input will direct where the crawler goes and how the collected data is analyzed. Web scraping replaces the need for manual data entry and more easily reveals trends among data collected. It can also aggregate information from multiple sources into one central location. While this application provides three specific examples of web crawling/scraping, it could be easily altered to better suit additional markets and/or needs.
Schmidt, Michael, "Web Crawler" (2015). All Capstone Projects. 148.