About web scraping
Web scraping, web harvesting or web data extraction is a tool we use to extract the data we need from websites. Web scraping software can access the World Wide Web directly using the Hypertext Transfer Protocol or a web browser. This application usually refers to automated processes executed using a bot or web crawler. It is a way of collecting specific data for later retrieval or analysis and then copying it from the web to a central local database or spreadsheet.
Scraping a web page involves retrieving it and extracting it. Available data can be stored for later use. Extraction may occur once obtained. You can analyze, search, re-format the content of a page, copy it to a spreadsheet, or upload it to a database. Web scanners usually take something from a page and take it elsewhere for a different purpose. Web scanners usually find and copy phone numbers, or companies and their URLs, or e-mail addresses (scratch contacts) on a web page.
Web scratches are used to scratch links and include web indexing, web mining and data mining, online price change monitoring and price comparison, product review scraping (for viewing competition), real estate listing, weather data monitoring, website changes Identify, do research, track online visits and reputation, integrate web mash up and web data.
Web pages are created using text-based markup languages (HTML and XHTML) and often contain a lot of useful data in the form of text. However, most web pages are designed for human end users, not for automated use. As a result, specialized tools and software have been developed to facilitate scraping web pages.
01 - Install Web scraping Software
Install Web scraping Software - ( We Get Webharvey Web Scraping Software as example ) below picture show how it work
after installation just license the software using utlicense file, (some computers requre download and install .net framework 3.5 , if requre that you must download and install that requre files) then open the application
03 - Finish the installation setup - Now ready to work
alright , now finish the basic installation setup now you can start the working with that software , lets see that above mention step in video
04 - Get Example Site for get data
all right now we going to get web data using web scraping , 1st we must open site ( which site you want to data extract) we take as example www.rainbowpages.lk to showing how get information from that site
We take as example Beauty Saloon
05 - Start Config
06 - Capture Data - Name Field
01 - Click On Business Name
02 - Click Capture Text
03 - Type Name ( which want name )
07 - Capture Address
01 - Click On Address field On Site
02 - Click Capture Text On small window
03 - Write Name on small window (Address)
08 - Capture Telephone Numbers
01 - Click on Telephone Numbers field
02 - Click Capture Text Field
03 - Type name ( Telephone number , phone .. any)
09 - Set Next Page Link
01 - Click 2nd Page Link On Site
02 - Click Set next Page Link
10 - Now Stop Config
Now Click Stop Config , that's it now finished the configuration of mind data collection. you now can mine all data that you configed
11 - Click On Start Mine
Then Comes All You Captured Data Like Bellow
that's finished , that all captured data appear here as above picture , now you can export it any format that they given. now you can use data for your any propose. ( use marketing, promotional, any other work )
