Effortless Form Submission Automation: A Comprehensive Guide with Selenium and Python
How to leverage Selenium and Python for efficient automation of online form submission.
Introduction
Testing user registration manually by filling out online forms continuously can be an annoying and tedious task as time goes on. In cases with large input data to test, data entry in forms can also be time-consuming. Hence the need to cut down on stress, automate this overwhelming task, and gain focus on the right things to do for growth.
One such tool that helps with the automation of form submission is Selenium webdriver.
Overview of Selenium and its role in web automation
Selenium WebDriver is an open-source tool that is used to efficiently automate web browsers. Its programmable interface supports interaction with web browsers like Chrome, Firefox, Edge, and others and allows testers and programmers to automate tasks that involve web interactions.
One amazing thing about this tool is its compatibility with a lot of popular programming languages such as Java, Python, C#, and JavaScript.
Some features of Selenium WebDriver include cross-browser compatibility, language support, element interaction, navigation, forms and user inputs, etc.
In this article, we are going to fill out an online form and submit it with the help of Selenium and Python scripts. We will also be analyzing how to handle the validation of successful form submission and how to resubmit a form when an error is experienced.
Setting up Your Environment
To follow up on this article, you need to set up your environment with the right tools.
- Download and install Python
Download Python according to your operating system and install the package in your system.
- Install Selenium WebDriver
Selenium WebDriver can be installed simply using the pip install manager. PIP is a package management system used to install/manage packages and libraries written in Python.
To do this, open your command line and enter this command pip install selenium
- Download and Install PyCharm
PyCharm is an IDE created specifically for Python. For this tutorial, we will be using PyCharm however, any IDE of your choice works fine too. To use PyCharm, you can download the community version.
- Install WebDriver Manager for the Python library
To wrap this section up, you need to install a WebDriver which will allow Selenium to interact with your browser. For this tutorial, we will use the WebDriver Manager for Python. The reason for this is that it simplifies the management of binary drivers for different browsers.
WebDriver Manager can be installed using the pip command below:
pip install webdriver-manager
Demonstrating how to fill a form with Python and Selenium
This section handles how to fill out an online form
Test Scenario
In this article, we will be doing the following
Open the Create an Account page on Twilio
Fill in the name section
Fill in the email address section
Fill in the password
Click on the agreement checkbox
Click on Start your free trial to submit the form
First: Open a web page on Chrome
The first thing you need to do to handle the test scenario above is to open the Create an Account Page. To do this, you need to set up and configure Selenium WebDriver for Google Chrome using Python.
First, Import the webdriver module from the Selenium library, Import the Service class from the Selenium Chrome package as ChromeService, and Import ChromeDriverManager from the webdriver_manager library.
from selenium import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService
from webdriver_manager.chrome import ChromeDriverManager
Next, Initialize the WebDriver with the ChromeDriverManager.
driver = webdriver.Chrome(service=ChromeService(ChromeDriverManager().install()))
The code above ensures that the appropriate version of the ChromeDriver executable is installed and used. This is a convenient way to handle ChromeDriver management within your Selenium tests.
Lastly, we need to navigate the WebDriver to the given URL we will be working with.
driver.get("https://www.twilio.com/try-twilio")
After executing this line, the WebDriver will load the specified webpage, and you can perform various actions, such as interacting with elements on the page or extracting information from it using Selenium.
Pro Tip. Use this command driver.maximize_window() after initializing your WebDriver to ensure you have a better view of the page and nothing obstructs your actions.
Second: Filling out Form Fields
After carrying out the first step, the next step is to fill out the form fields. To do this, we need an understanding of two critical factors
Identification of form elements with the various Selenium locators
How to input data into the form fields
To identify the various form elements of this page, we need to first understand Selenium locators. There are several locators available in Selenium. They include ID, XPATH, CSS_Selector, Classname, name, linkText, partial_link_text, and tagname.
In this article, we will be focusing on using CSS_Selector; however, toolsqa has an exhaustive blog on selenium locators.
The first element we will examine is the First Name field. To analyze this, you need to right-click on it, a list of options pops up, scroll down, and click on Inspect option. The image below should be on your screen now.
The part highlighted in the screenshot contains the details of the First name element and it can be found below
<input type="text" id="FirstName" name="FirstName" class="material-form-control sl_whiteout" tabindex="1" maxlength="255">
Next, choose the selenium selector which will be used to automate this element. As stated earlier, in this article, we will be using CSS_Selector for all elements. This article by Sauce Labs explains CSS_Selectors in detail.
Using the CSS_Selector, we can see that the ID attribute has a value FirstName so the CSS_Seelctor used in locating this element will be #FirstName.
Before we can be able to locate this element, we need to import another important class (By). Selenium uses the By class in complement to the find_element or find_elements method to easily locate its selectors and the element.
To import this class, use the code below
from selenium.webdriver.common.by import By
Now we will be using the driver instance to call the find-element method, the By class with its associated selector, and inform the value of the CSS_Selector we are looking for which in this case is #FirstName and finally, store it in a first_name variable. The code should be as below
first_name = driver.find_element(By.CSS_SELECTOR, "#FirstName")
After locating the element, the next step is to send our data into the required field. To achieve this, we will be using the send_keys method. In this case, we want to enter the text fname into the First name field. So the code looks as below
first_name.send_keys("fname")
We have successfully automated the filling of the First Name field. Using the same methods as the First Name field, we will be filling the Last Name, Email, and Password fields respectfully.
For the Last name field, right-click on it and click the inspect option. The highlighted part of the image below shows the element.
<input type="text" id="LastName" name="LastName" class="material-form-control sl_whiteout" tabindex="2" maxlength="255" xpath="1">
last_name = driver.find_element(By.CSS_SELECTOR, "#LastName")
Using the send_keys method, enter the data you would want to exist on the field
last_name.send_keys("lname")
Next is the Email field. The highlighted part of the image below shows the element for the Email field
email = driver.find_element(By.CSS_SELECTOR, "#EmailAddr")
Using the send_keys method, enter the data you would want to exist on the field
email.send_keys("randomemail@gmail.com")
Next is the Password field. The highlighted part of the image below shows the element for the password field.
password = driver.find_element(By.CSS_SELECTOR, "#Passwd")
Using the send_keys method, enter the data you would want to exist on the field.
password.send_keys("pasword1234+")
We have completed filling our data into the fields. Now the next step is to tick the acceptance of Twilo terms and service.
This is a checkbox and we can also automate this action using selenium. To automate this, we right-click the checkbox and click on the inspect option. The image below displays the element of the checkbox from which we will see its properties and decide the right selector to use.
<input type="checkbox" class="big-checkbox" id="Tos" name="Tos" tabindex="7" xpath="1">
We will be using the CSS_Selector to locate this checkbox. Using the CSS_Selector, we can see that the ID attribute has a value Tos so the CSS_Selector used in locating this element will be #Tos.
checkbox = driver.find_element(By.CSS_SELECTOR, "#Tos")
Unlike the form fields where we use the send_keys method to enter our data. For checkboxes, we use the click method to tick the box as affirmative.
checkbox.click()
Third: Submitting forms
The last step is to submit our filled form.
To do this, we right-click on the Start your free trial button and click on the inspect option. The highlighted part of the image below displays the properties of this element.
<button id="signup-button" type="submit" class="m-button" data-loading-text="<i class='icon icon-spinner icon-spin'></i>" tabindex="10" xpath="1">Start your free trial</button>
Using the CSS_Selector and the steps followed for the checkbox tick, we will have the code below
submit = driver.find_element(By.CSS_SELECTOR, "#signup-button")
submit.click()
Now, it’s time to run our code and see our form automatically submitted.
Dealing with Form Validation
With Selenium, it is possible to know the status of your form submission. We will be discussing this broadly in this section
Handling form validation messages and errors
When filling out a form online, it is common for websites to perform validation on the submitted data. If there are errors, the website might display error messages and if there are no errors, a success note will be displayed. Selenium can be used to locate and read these error messages.
We will be using the same form above for this illustration. If we use the data we entered previously, an error will pop up underneath the password field stating that the password doesn’t meet the requirement and this will hinder the submission of our form. We will be writing lines of code to ensure Selenium picks up error messages like this and prints it out on the console when the code is run.
To ensure Selenium does just this, right-click the error message Password is too weak and click on the Inspect option. The properties of the element are displayed and you look for the best Selenium locator for it. For this example, we will be using the CSS_Selector with the value span[for='Passwd'] ul li.
Unlike the previous demonstrations above, this particular action requires a couple of methods for it to be executed.
Firstly, we need to import the WebDriverwait constructor. This constructor takes a webdriver instance and timeout in seconds.
from selenium.webdriver.support.ui import WebDriverWait
Next, we also need to import the expected_conditions module from the Selenium webdriver library and rename it as EC for easy use.
from selenium.webdriver.support import expected_conditions as EC
The expected_conditions module contains various conditions that can be used in complement with Selenium’s webdriverWait for specific conditions to be met on the web page.
Pro Tip: To make your code look neater, attach the code above to the previous constructors imported in the early part of the article.
After importing the necessary constructors and modules, we will use the code below to get the error message printed in the console.
wait = WebDriverWait(driver, 5)
successTextElement = wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "span[for='Passwd'] ul li")))
successText = successTextElement.text
print(successText)
The code does the following:
The wait = WebDriverWait(driver,5) creates a WebDriverWait object which is used to wait for a certain condition to be met before proceeding, and in this case, it is to wait for 5 seconds for a specific element to become present on the page.
EC.presence_of_element_located((By.CSS_SELECTOR, “span[for=’Passwd’]ul li”)) is the expected condition that the WebDriverWait will wait for. It simply checks for the presence of an element with the CSS_SELECTOR span[for=’Passwd’]ul li.
wait.until(....) waits for the element to be present and assigns it to the variable successTextElement.
successText = successTextElement.text extracts the text content of the successTextElement and assigns it to the variable successText.
Lastly, print(successText) prints the extracted text to the console.
The image below shows the output on the console
Writing custom validation checks
You can also write custom validation checks to verify if the form was submitted successfully or if an error message is displayed. Using the last illustration, let us further analyze how to write a custom validation check.
To verify the success of your form submission, using custom validation checks, use the code below.
if "success" in driver.current_url:
print("Form submitted successfully!")
else:
print("Form submission failed.")
The code does the following
Check if the string “success” appears in the URL of the web page. If it does, it prints “Form submitted successfully!” and if it doesn’t, it prints “Form Submission failed.”
Automating form resubmission with corrected data
If you encounter validation errors when trying to submit your form, you may need to resubmit the form with the corrected data, just like our form which was not submitted because it failed to meet the password requirement. To resubmit the form, you can do this by first checking for errors, clearing the form fields, entering the corrected data, and then resubmitting the form.
In this example, we are going to enter new data into every field of the form and enter a password that meets the requirement before clicking on the agreement button and Starting a free trial. Input the code below after the submit.click() and this should do the trick.
# Check for validation errors
wait = WebDriverWait(driver, 5)
error_messages = wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "span[for='Passwd'] ul li")))
if error_messages:
print("Form submission failed. Correcting data...")
# Clear the form fields
first_name.clear()
last_name.clear()
email.clear()
password.clear()
# Enter corrected data
first_name.send_keys("new_username")
last_name.send_keys("new_password")
email.send_keys("new_email@gmail.com")
password.send_keys("new_pasword12345+")
# Resubmit the form
submit.click()
else:
print("Form submitted successfully!")
The code above checks for validation errors clears the form fields, enters the corrected data, and resubmits the form.
Conclusion
In this tutorial on automating form filling with Selenium Python, we delved into understanding why automation is important and an overview of Selenium WebDriver. We further discussed the several software required to set up and start automating forms.
Furthermore, we demonstrated how to fill out a form online using Selenium and Python.
One key aspect we highlighted in detail was dealing with several form validations and how to automate form resubmission.
Additional Resources
Here are some resources that can help you understand how to automate the filling of forms with Selenium and Python.