Introduction
This is an introductory article and a navigator article for the SearchResultParser project/web tool. This series of articles will cover the step-by-step creation of this tool.
The main idea of the SearchResultParser project is to parse data from search results (not just Google). In general, this tool will support the following search engines and machines (not all at start, but nevertheless):
- Yahoo
- Bing
- DuckDuckGo
- Baidu
- Yandex
- Aol
- StackOverflow
- GitHub
- Ask
- YouTube
- MyAnimeList
- GoogleScholar
- GoogleNews
- Coursera
I will delegate the main parsing work to this python library - search-engine-parser. The main one, but not all. I will write the Google parser and everything that won't work via search-engine-parser by myself. All data and source code will be available on GitHub.
Projects tech stack
As a frontend, I chose ReactJS + MaterialUI library. Why them? Why not. React is considered the most popular JS framework and it would be a sin on my part not to touch it. Also, as an additional flexibility in styling and tuning the site's design, I will use TailwindCSS.
For the backend, I always have one answer - Django. I like this framework, I like its ideology and logic. Plus, it's the only one I know (^///^)
As a hosting provider, reg.ru will act as my hosting provider. I would like to study other hosting providers, except for beget.
The monetization model of the site will be based on the so-called freemium + advertising networks. I am going to make all payments through Robokassa. And the advertising network will be from Yandex, Direct.
What you can learn from it
This series of articles implies that I will write this site from scratch to the end and describe the entire process in detail. And to be more precise, in this series you will be able to learn such things as:
- How to use Django and React together
- How to combine TailwindCSS together with React
- How to publish a website on Reg.ru host provider
- How to sync changes made on the developer website to already published
- How to use proxy with parsers
- How to make user-agent swapping possible
- How to make your python scrapper in parallel manner
Navigation
- News about a new project
- Integration a React app into Django project
- Developing base websites user interface(frontend)
- Implementing interactive tutorial for the website
- Adding a user authentication on the website
- Adding support for several languages
- How to use and setup REST framework with Django
- How to publish a website on a Reg.ru host provider
Conclusions
I will write the conclusion when I finish the project. (∪.∪ )...zzz