Python script to login to website automatically without selenium

I am trying to monitor various investments I hold through a single database (mysql with an MS Access Frontend). I have sourced solution for most of them as can source the unit price / share price without needing to login to a portal. My issue is for one of my investments I can only find out the value by logging in.

I have got this working through my laptop using Selenium but have my raspberry pi doing all the harvesting and Selenium is flakey on it and keeps breaking when I do updates etc.

I am hoping someone may be able to direct me on what python module to use for performing the same without selenium. For my other harvesting I am using a mix of requests_html & urllib.request so slightly have them as a preference but couldn't work it out. Have also attempted mechanize but no luck.

I have created a test account on the website and have those details in the code however it doesn't allow access to what is behind the login but my issue is being able to submit the login details in the first place.

from selenium import webdriver
import time

browser = webdriver.Edge(r"C:\Python\edgedriver\msedgedriver.exe")
browser.get('https://app.raizinvest.com.au/login')

time.sleep(4)  
browser.find_element_by_name("email").send_keys("")
browser.find_element_by_name("password").send_keys("Test4321")
browser.find_element_by_xpath("/html/body/div/div/div[1]/div/div/div/form/div[4]/button").click()
time.sleep(3)
print(str(browser.find_elements_by_class_name("page-content__banner-account-value")[0].text))
browser.close()

asked Jul 3, 2021 at 8:15

Your selenium code keeps on breaking cause you are using xpath like this

/html/body/div/div/div[1]/div/div/div/form/div[4]/button -

you can even introduce explicit waits in your code to have more reliability.

Helium is one such tool which is bit advance than selenium but have it's own limitation such as will work only in chrome and FF and with Python itself.

If you are interest in Selenium Python bindings, I would re-write your code as below :

driver.maximize_window()
driver.get("https://app.raizinvest.com.au/login")
wait = WebDriverWait(driver, 10)
wait.until(EC.element_to_be_clickable((By.NAME, "email"))).send_keys('')
wait.until(EC.element_to_be_clickable((By.NAME, "password"))).send_keys('Test4321')
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[type='submit']"))).click()

Imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Selenium - Python - Explicit waits

Python - Helium - Installation

Python - Helium - API's

answered Jul 3, 2021 at 8:36

cruisepandeycruisepandey

27.5k6 gold badges17 silver badges37 bronze badges

5

Yes, you read it right. You can log in to any website automatically using Python. There is much automation technique you can do using Python. Have you ever got irritated for logging in to the websites you often visit?. So here comes the automation technique to make automated login. So, you don’t need to enter your username or email and password anymore. Except at the setting up! Let’s just jump on the topic.

Since it’s just simple automation which doesn’t contain much code and doesn’t require coding knowledge. I’ll explain the code in a much easier way. The requirements are

• Python 3
• chrome driver

Chrome driver is nothing but a browser which is chromium project made for automation. You can download chrome driver here. Download the chrome driver version as the same as your Google Chrome. You need to install a python package to handle chrome driver. You can install it using pip command. The command is

pip install selenium

Here for this tutorial, I’m going to use twitter for automated login. Now create a new folder somewhere and open any IDE of your choice and paste the below code.

from selenium import webdriver
import timedriver = webdriver.Chrome(executable_path="chromedriver.exe")
driver.get('https://twitter.com/login')
time.sleep(6)
driver.find_element_by_xpath('//*[@id="react-root"]/div/div/div[2]/main/div/div/form/div/div[1]/label/div/div[2]/div/input').send_keys('YOUR_USERNAME')
driver.find_element_by_xpath('//*[@id="react-root"]/div/div/div[2]/main/div/div/form/div/div[2]/label/div/div[2]/div/input').send_keys('YOUR_PASSWORD')
driver.find_element_by_xpath('//*[@id="react-root"]/div/div/div[2]/main/div/div/form/div/div[3]/div/div').click()

Place your chrome driver in the same folder where this python file is. Let’s breakdown the code. At 1st line, from selenium Python package we’re importing web driver to handle the chrome driver browser. At 2nd line time package is imported. The path of the chrome web driver is mentioned in the 3rd line. At next line, we’ve provided the twitter’s login URL which will directly take you to the login page of twitter. Then time.sleep(6) line is used to suspend the execution of the program for a given amount of time. Here, 6 means that the program is going to sleep of 6 seconds and after 6 seconds it’ll resume the following code. Actually sleep function is used because the website needs some time to load that’s the main reason for using sleep function in the program. Next line, driver.find_element_by_xpath(‘//*[@id=”react-root”]/div/div/div[2]/main/div/div/form/div/div[1]/label/div/div[2]/div/input’).send_keys(‘YOUR_USERNAME’) is used to find the username input element in twitter login page. Fill your username or email or phone in place of YOUR_USERNAME. Then in the next line of the code is used to autofill the password field. In place of YOUR_PASSWORD fill your password and that’s it! This will the last time you’ll enter your password for the specified website. The last line is to click the login button on that page. The program will automatically click the login button. Now you can run this program. In the code, I have used find_element_by_xpath() function to find the input element and button element. This XPath will vary for the different websites and for different elements so you have to find the required element’s XPath. This can be done in your chrome browser by hovering the mouse on that element and inspecting that element. after inspecting right click on that element and hover over copy option and select copy XPath, not the Full XPath and paste it in the code.

The above image depicts on copying the XPath of the username. The same process is done for all elements. By using this thing you can automate login for any website. If getting stuck in some issues please let me know in the response section. I’ll try helping you. Start building your own login automation. Not only login automation you can also do much automation using selenium and chrome driver.

Disclaimer: The Blog Content has been made available for informational and educational purposes only. I hereby disclaim any and all liability to any party for any direct, indirect, implied, punitive, special, incidental or other consequential damages arising directly or indirectly from any use of the Blog Content is solely responsible by the readers

Can Python automate website interaction?

How do you enter data into a website using Python?

How do I automate a website login?