近年来，随着互联网的发展，有效获取互联网信息的手段已成为无数互联网公司

In recent years, with the development of the Internet, effective means of obtaining Internet information has become a new research field for countless Internet companies. In an era of data richness, those who can get more useful data can get more benefits. Web crawlers are the most commonly used method for obtaining data on the Internet. <br>Web crawlers are also called web robots and web spiders. It is an information program that can automatically collect information on the Internet according to a given URL according to certain rules. This subject implements a web crawler program system based on Python. In this paper, we basically discussed some of the main problems in the process of implementing crawlers, such as: how to use Python for simulated login, how to obtain information to match strings through regular expressions, and use MySQL to store data. <br>Through this crawler program, you can easily collect some data information of the Douban website, such as: dynamic information on the homepage of the website, movie details and movie reviews. <br>Keywords: Python; regular expressions; MySQL; web crawler

In recent years, with the development of the Internet, effective access to Internet information has become a new research field for countless Internet companies. In an age of data-rich, those who can get more useful data can make more money, and web crawlers are the most common way to get data on the Internet.<br> Network crawlers are also known as webrobots and web spiders, which are programs that automatically collect information on the Internet according to a given URL according to certain rules. This project implements a Python-based network reptile program system. In this paper, we basically discuss some of the main problems in implementing the reptile, such as how to use Python for simulated logins, how to get information to match strings through regular expressions, and using MySQL to store data.<br> Through this crawler, it is easy to collect some data information about the Bean flap website, such as: dynamic news on the home page of the website, movie details and film reviews.<br>Keywords: Python; Regular Expressions; MySQL; Web Crawler

In recent years, with the development of the Internet, effective access to Internet information has become a new research field of numerous Internet companies. In an era of rich data, those who can get more useful data can get more profits. Web crawler is the most commonly used method to get data on the Internet.<br>Web crawler is also known as web robot and web spider. It is a program that can automatically collect information on the Internet according to a given URL according to certain rules. In this paper, a web crawler system based on Python is implemented. In this paper, we basically discussed some main problems in the process of implementing crawler, such as how to use Python to simulate login, how to get information through regular expression to match strings, and how to use Mysql to store data.<br>Through this crawler, you can easily collect some data information of Douban website, such as: dynamic information of website homepage, movie details, movie reviews, etc.<br>Keywords: Python; regular expression; MySQL; web crawler