Build Fuel Price Tracker Using Python
Last Updated :
03 Jan, 2023
In this modern-day lifestyle, fuel has become a necessity for all human beings. It is a base for our life-style. So, we are going to write a script to track their price using Python.
Modules Needed
- bs4: Beautiful Soup(bs4) is a Python library for pulling data out of HTML and XML files. This module does not come built-in with Python. To install this type the below command in the terminal.
pip install bs4
- requests: Request allows you to send HTTP/1.1 requests extremely easily. This module also does not come built-in with Python. To install this type the below command in the terminal.
pip install requests
Let’s see the stepwise execution of the script
Step 1: Import all dependence
Python3
import pandas as pd
import requests
from bs4 import BeautifulSoup
|
Step 2: Create a URL get function
Python3
def getdata(url):
r = requests.get(url)
return r.text
|
Step 3: Now pass the URL into the getdata() function and Convert that data into HTML code
Python3
soup = BeautifulSoup(htmldata, 'html.parser' )
result = soup.find_all( "div" , class_ = "gold_silver_table" )
print (result)
|
Output :
[<div class=”gold_silver_table”> <table border=”0″ cellpadding=”1″ cellspacing=”1″ width=”100%”> <tr class=”first”> <td class=”heading” width=”200″>City</td> <td class=”heading” width=”200″>Today Price</td> <td class=”heading” width=”200″>Yesterday’s Price</td> </tr> <tr class=”even_row”> <td><a href=”/petrol-price-in-new-delhi.html” title=”New Delhi”>New Delhi</a></td> <td> ₹ 82.08</td> <td> ₹ 82.03</td> </tr> <tr class=”odd_row”> <td><a href=”/petrol-price-in-kolkata.html” title=”Kolkata”>Kolkata</a></td> <td> ₹ 83.57</td> <td> ₹ 83.52</td> </tr> <tr class=”even_row”> <td><a href=”/petrol-price-in-mumbai.html” title=”Mumbai”>Mumbai</a></td> <td> ₹ 88.73</td> <td> ₹ 88.68</td> </tr> <tr class=”odd_row”> <td><a href=”/petrol-price-in-chennai.html” title=”Chennai”>Chennai</a></td> <td> ₹ 85.04</td> <td> ₹ 85.00</td> </tr> <tr class=”even_row”> <td><a href=”/petrol-price-in-gurgaon.html” title=”Gurgaon”>Gurgaon</a></td> <td> ₹ 79.92</td> <td> ₹ 79.84</td> </tr> <tr class=”odd_row”> <td><a href=”/petrol-price-in-noida.html” title=”Noida”>Noida</a></td> <td> ₹ 82.23</td> <td> ₹ 82.30</td> </tr> <tr class=”even_row”> <td><a href=”/petrol-price-in-bangalore.html” title=”Bangalore”>Bangalore</a></td> <td> ₹ 84.75</td> <td> ₹ 84.70</td> </tr> <tr class=”odd_row”> <td><a href=”/petrol-price-in-bhubaneswar.html” title=”Bhubaneswar”>Bhubaneswar</a></td> <td> ₹ 82.47</td> <td> ₹ 82.59</td> </tr> <tr class=”even_row”> <td><a href=”/petrol-price-in-chandigarh.html” title=”Chandigarh”>Chandigarh</a></td> <td> ₹ 78.96</td> <td> ₹ 78.92</td> </tr> <tr class=”odd_row”> <td><a href=”/petrol-price-in-hyderabad.html” title=”Hyderabad”>Hyderabad</a></td> <td> ₹ 85.30</td> <td> ₹ 85.25</td> </tr> <tr class=”even_row”> <td><a href=”/petrol-price-in-jaipur.html” title=”Jaipur”>Jaipur</a></td> <td> ₹ 90.08</td> <td> ₹ 89.24</td> </tr> <tr class=”odd_row”> <td><a href=”/petrol-price-in-lucknow.html” title=”Lucknow”>Lucknow</a></td> <td> ₹ 82.20</td> <td> ₹ 82.09</td> </tr> <tr class=”even_row”> <td><a href=”/petrol-price-in-patna.html” title=”Patna”>Patna</a></td> <td> ₹ 84.73</td> <td> ₹ 84.88</td> </tr> <tr class=”odd_row”> <td><a href=”/petrol-price-in-trivandrum.html” title=”Trivandrum”>Trivandrum</a></td> <td> ₹ 83.91</td> <td> ₹ 84.03</td> </tr> </table> </div>]
Note: These scripts will give you only Raw data in String format you have to print your data with your needs.
Step 4: Now, Search your needed data into string with soup.find_all().
Python3
mydatastr = ''
result = []
for table in soup.find_all( 'tr' ):
mydatastr + = table.get_text()
mydatastr = mydatastr[ 1 :]
itemlist = mydatastr.split( "\n\n" )
for item in itemlist[: - 5 ]:
result.append(item.split( "\n" ))
result
|
Output :
Step 4: Make a DataFrame for displaying your result.
Python3
df = pd.DataFrame(result[: - 8 ])
df
|
Complete code:
Python3
import requests
import pandas as pd
from bs4 import BeautifulSoup
def getdata(url):
r = requests.get(url)
return r.text
soup = BeautifulSoup(htmldata, 'html.parser' )
mydatastr = ''
result = []
for table in soup.find_all( 'tr' ):
mydatastr + = table.get_text()
mydatastr = mydatastr[ 1 :]
itemlist = mydatastr.split( "\n\n" )
for item in itemlist[: - 5 ]:
result.append(item.split( "\n" ))
df = pd.DataFrame(result[: - 8 ])
df
|
Output :
Share your thoughts in the comments
Please Login to comment...