#datastoryteller

After the amount of data being collected from Coding and YouTube API; the data needs to make sense to people if it’s to lead to better engagement and increased conversions. That’s where visualization comes and called as Data storytelling.

Data storytelling is the process of combining graphics and narratives to help audiences understand complex data.

I'll explore few YouTube channels data and to provide as visualization; it make sense to people if it’s to lead to better engagement and increased conversions.

#playwright #Python #textcapture #stackoverflow

solution: BELOW

Helium is an open source framework for web-based automation. Its full name is selenium Python helium. As can be seen from its name, helium seems to be closely related to selenium. Helium is much more high-level. In Selenium, you need to use HTML IDs, XPaths and CSS selectors to identify web page elements. Helium on the other hand lets you refer to elements by user-visible labels.

As a result, Helium scripts are typically 30-50% shorter than similar Selenium #scripts. Currently, #helium only supports Chrome and Firefox. I have made specifying web automation cases by using helium as simple as describing them to someone looking as well as scraping utility by using with #requests_HTML to get results. Most of them followed Selenium based code in helium; whereas can simplified then selenium. I have written very basic and not applied any complex code/beautify the same.

I am usually using data #scraping for the data #analytics; The main purpose of helium is scraping, capture data which is based on JS based sites and more on that.

Sample code: https://github.com/katesanalyst/kates...

Documentation: https://selenium-python-helium.readth... https://github.com/mherrmann/selenium... "

Flipkart is gearing up for the mega shopping festival named – Flipkart Big Billion Days. You will be able to purchase a wide range of gadgets, accessories, fashion items and much more from October 3 to October 10 as part of the Flipkart Big Billion Days. Flipkart has revealed that the sale is powered by lakhs of sellers, artisans, and brands across India.

However, explore prices rather than regular/normal sale. Therefore, randomly selected few product categories and will check after Big Billion Days then reveal actual benefits for consumer as well as Flipkart.

Web Scraping data from Flipkart using Python for Big Billion deals

Today, the Internet is flooded with an enormous amount of data relative to what we had a decade ago. The amount of data we produce every day is truly mind-boggling.

Web Scraping!?

Web Scraping is important because, irrespective of the industry, the web contains information that can provide actionable insights for businesses to gain an advantage over competitors.

Web scraping, also known as web data extraction, is the process of retrieving or “scraping” data from a website. This information is collected and then exported into a format that is more useful for the user and it can be a spreadsheet or an API. Although web scraping can be done manually, in most cases, automated tools are preferred when scraping web data as they can be less costly and work at a faster rate.

Is Web Scraping Legal?

The simplest way is to check the robots.txt file of the website. You can find this file by appending “/robots.txt” to the URL that you want to scrape. It is usually at the website domain /robots.txt. If all the bots indicated by ‘user-agent: *’ are blocked/disallowed in the robots.txt file, then you’re not allowed to scrape. For this article, I am scraping the Flipkart website. So, to see the “robots.txt” file, the URL is www.flipkart.com/robots.txt

What are the Steps Involved in Web Scraping?

To fetch the data using Web Scraping using Python, we need to go through the following steps:

Find the URL that you want to scrape.

Inspecting the Page and website.

Find the ways to get data; some of them using API or JS etc.

Find the suitable and simple code approach.

Find the data you want to extract.

Write the code.

Run the code & extract the data.

Finally, Store the data in the required format.

Packages used for Web Scraping

Following python packages are used:

Pandas: Pandas is a library used for data manipulation and analysis. It is used to store the data in the desired format.

Requests-HTML: This library intends to make parsing HTML (e.g. scraping the web) as simple and intuitive as possible.

Benefits of Data and Analytics as consumer for Flipkart.

1. Price of your products according to regular vs big billion days.

2. What customers select for what products; based on customer demands.

Simple enough, BENEFITS for your purchases regular vs big billion days; save every penny as well.

Whereas client or competitor;

1. You can price your products according to demand and by what customers expect.

2. Manage your inventory better by anticipating customer demands.

3. Customers can select from a combination of features or service that meets their needs.

4. Knowing what products will sell when and where allows manufacturing efforts to anticipate demand and prevent a loss of sales.

5. More competitive prices, reduced costs, and higher sales are results of targeted marketing efforts.

Simple enough, REVENUE is your top line and you’re probably tracking this for your accountant already.

I have selected few categories and will keep update what will benefits after big billion days.

This is purely as an educational purpose; would like to know any clarification on the same. It just 90+ line of code to expelled data as well. I have not shared any code and data.