Selenium with Python: Complete Automation Guide

Selenium remains one of the most popular browser automation frameworks. Combined with Python's simplicity, it creates a powerful toolkit for web testing and automation. Let's explore how to master Selenium with Python.

Why Selenium + Python?

• 🐍 Python's Simplicity: Easy to learn and write
• 🌐 Cross-Browser Support: Chrome, Firefox, Safari, Edge
• 📦 Rich Ecosystem: Pytest, unittest, behave integration
• 👥 Large Community: Extensive resources and support
• 🔧 Mature Framework: Battle-tested and reliable

Installation and Setup

bash
1# Install Selenium
2pip install selenium
3
4# Install WebDriver Manager (automatic driver management)
5pip install webdriver-manager
6
7# Optional: Install pytest for testing
8pip install pytest pytest-html

Your First Selenium Script

python
1from selenium import webdriver
2from selenium.webdriver.common.by import By
3from selenium.webdriver.chrome.service import Service
4from webdriver_manager.chrome import ChromeDriverManager
5
6# Setup WebDriver
7driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()))
8
9try:
10    # Navigate to website
11    driver.get('https://example.com')
12    
13    # Find element and interact
14    heading = driver.find_element(By.TAG_NAME, 'h1')
15    print(f"Page heading: {heading.text}")
16    
17    # Click a link
18    link = driver.find_element(By.LINK_TEXT, 'More information')
19    link.click()
20    
21finally:
22    # Always close the browser
23    driver.quit()

Finding Elements: All Methods

Selenium provides multiple ways to locate elements:

python
1from selenium.webdriver.common.by import By
2
3# By ID
4driver.find_element(By.ID, 'username')
5
6# By Name
7driver.find_element(By.NAME, 'email')
8
9# By Class Name
10driver.find_element(By.CLASS_NAME, 'btn-primary')
11
12# By Tag Name
13driver.find_element(By.TAG_NAME, 'h1')
14
15# By Link Text
16driver.find_element(By.LINK_TEXT, 'Sign In')
17
18# By Partial Link Text
19driver.find_element(By.PARTIAL_LINK_TEXT, 'Sign')
20
21# By CSS Selector (most flexible)
22driver.find_element(By.CSS_SELECTOR, 'button.submit-btn')
23driver.find_element(By.CSS_SELECTOR, '#login-form input[type="password"]')
24
25# By XPath (most powerful)
26driver.find_element(By.XPATH, '//button[@class="submit"]')
27driver.find_element(By.XPATH, '//div[@id="content"]//p[1]')
28
29# Find multiple elements
30elements = driver.find_elements(By.CLASS_NAME, 'product-card')

Interacting with Elements

Basic Interactions

python
1from selenium.webdriver.common.keys import Keys
2
3# Typing text
4email_field = driver.find_element(By.ID, 'email')
5email_field.send_keys('user@example.com')
6
7# Clearing text
8email_field.clear()
9
10# Clicking
11submit_button = driver.find_element(By.ID, 'submit')
12submit_button.click()
13
14# Keyboard actions
15search_box = driver.find_element(By.NAME, 'q')
16search_box.send_keys('Python Selenium')
17search_box.send_keys(Keys.RETURN)  # Press Enter
18
19# Checkboxes and radio buttons
20checkbox = driver.find_element(By.ID, 'terms')
21if not checkbox.is_selected():
22    checkbox.click()
23
24# Dropdowns
25from selenium.webdriver.support.select import Select
26
27dropdown = Select(driver.find_element(By.ID, 'country'))
28dropdown.select_by_visible_text('United States')
29dropdown.select_by_value('us')
30dropdown.select_by_index(1)
31
32# Get element attributes
33link_url = driver.find_element(By.TAG_NAME, 'a').get_attribute('href')
34is_visible = element.is_displayed()
35is_enabled = element.is_enabled()

Waits: The Key to Stable Tests

Explicit Waits (Recommended)

python
1from selenium.webdriver.support.ui import WebDriverWait
2from selenium.webdriver.support import expected_conditions as EC
3
4# Wait up to 10 seconds for element to be clickable
5wait = WebDriverWait(driver, 10)
6element = wait.until(
7    EC.element_to_be_clickable((By.ID, 'submit-button'))
8)
9element.click()
10
11# Common wait conditions
12wait.until(EC.presence_of_element_located((By.ID, 'content')))
13wait.until(EC.visibility_of_element_located((By.CLASS_NAME, 'modal')))
14wait.until(EC.invisibility_of_element_located((By.ID, 'loading')))
15wait.until(EC.title_contains('Dashboard'))
16wait.until(EC.url_contains('/dashboard'))
17
18# Wait for element to contain text
19wait.until(
20    EC.text_to_be_present_in_element(
21        (By.ID, 'status'),
22        'Complete'
23    )
24)
25
26# Custom wait condition
27wait.until(lambda driver: len(driver.find_elements(By.CLASS_NAME, 'item')) > 5)

Implicit Wait

python
1# Set implicit wait for all find operations
2driver.implicitly_wait(10)  # Wait up to 10 seconds
3
4# Now all find_element calls will wait
5element = driver.find_element(By.ID, 'dynamic-content')

Page Object Model (POM)

Organize your test code with Page Objects:

python
1# pages/login_page.py
2from selenium.webdriver.common.by import By
3from selenium.webdriver.support.ui import WebDriverWait
4from selenium.webdriver.support import expected_conditions as EC
5
6class LoginPage:
7    # Locators
8    EMAIL_INPUT = (By.ID, 'email')
9    PASSWORD_INPUT = (By.ID, 'password')
10    SUBMIT_BUTTON = (By.CSS_SELECTOR, 'button[type="submit"]')
11    ERROR_MESSAGE = (By.CLASS_NAME, 'error-message')
12    
13    def __init__(self, driver):
14        self.driver = driver
15        self.wait = WebDriverWait(driver, 10)
16    
17    def open(self):
18        self.driver.get('https://example.com/login')
19        return self
20    
21    def enter_email(self, email):
22        element = self.wait.until(
23            EC.presence_of_element_located(self.EMAIL_INPUT)
24        )
25        element.clear()
26        element.send_keys(email)
27        return self  # Allow method chaining
28    
29    def enter_password(self, password):
30        element = self.driver.find_element(*self.PASSWORD_INPUT)
31        element.clear()
32        element.send_keys(password)
33        return self
34    
35    def click_submit(self):
36        element = self.wait.until(
37            EC.element_to_be_clickable(self.SUBMIT_BUTTON)
38        )
39        element.click()
40    
41    def get_error_message(self):
42        element = self.wait.until(
43            EC.visibility_of_element_located(self.ERROR_MESSAGE)
44        )
45        return element.text
46    
47    def login(self, email, password):
48        """Convenience method for complete login"""
49        self.enter_email(email).enter_password(password).click_submit()
50
51# Usage in test
52def test_login():
53    driver = webdriver.Chrome()
54    login_page = LoginPage(driver)
55    
56    login_page.open().login('user@example.com', 'password123')
57    
58    # Verify login success
59    assert 'dashboard' in driver.current_url
60    driver.quit()

Advanced Techniques

Handling Multiple Windows/Tabs

python
1# Open new tab
2driver.execute_script("window.open('');")
3
4# Switch between windows
5original_window = driver.current_window_handle
6all_windows = driver.window_handles
7
8for window in all_windows:
9    if window != original_window:
10        driver.switch_to.window(window)
11        # Do something in new window
12        driver.close()
13
14# Switch back
15driver.switch_to.window(original_window)

Handling Alerts

python
1from selenium.webdriver.support import expected_conditions as EC
2
3# Wait for alert
4wait.until(EC.alert_is_present())
5
6# Switch to alert
7alert = driver.switch_to.alert
8
9# Get alert text
10print(alert.text)
11
12# Accept alert
13alert.accept()
14
15# Dismiss alert
16alert.dismiss()
17
18# Enter text in prompt
19alert.send_keys('My input')
20alert.accept()

Handling Frames

python
1# Switch to frame by index
2driver.switch_to.frame(0)
3
4# Switch to frame by name or ID
5driver.switch_to.frame('frame-name')
6
7# Switch to frame by WebElement
8frame_element = driver.find_element(By.ID, 'iframe1')
9driver.switch_to.frame(frame_element)
10
11# Switch back to main content
12driver.switch_to.default_content()

JavaScript Execution

python
1# Execute JavaScript
2driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
3
4# Click element with JavaScript (bypass overlays)
5element = driver.find_element(By.ID, 'hidden-button')
6driver.execute_script("arguments[0].click();", element)
7
8# Get data from page
9page_data = driver.execute_script("return document.getElementById('data').textContent;")
10
11# Wait for page load
12driver.execute_script("return document.readyState") == "complete"

Taking Screenshots

python
1# Full page screenshot
2driver.save_screenshot('screenshot.png')
3
4# Element screenshot
5element = driver.find_element(By.ID, 'chart')
6element.screenshot('chart.png')
7
8# Screenshot on test failure
9def test_example():
10    try:
11        # Test code
12        assert False
13    except AssertionError:
14        driver.save_screenshot('test_failed.png')
15        raise

Pytest Integration

python
1# conftest.py
2import pytest
3from selenium import webdriver
4from webdriver_manager.chrome import ChromeDriverManager
5
6@pytest.fixture
7def driver():
8    """Provide WebDriver instance"""
9    driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()))
10    driver.implicitly_wait(10)
11    driver.maximize_window()
12    yield driver
13    driver.quit()
14
15# test_login.py
16def test_successful_login(driver):
17    login_page = LoginPage(driver)
18    login_page.open()
19    login_page.login('valid@email.com', 'password123')
20    
21    assert 'dashboard' in driver.current_url
22
23def test_invalid_login(driver):
24    login_page = LoginPage(driver)
25    login_page.open()
26    login_page.login('invalid@email.com', 'wrong')
27    
28    assert 'Invalid credentials' in login_page.get_error_message()
29
30# Run tests
31# pytest test_login.py --html=report.html

Parallel Testing

python
1# conftest.py
2import pytest
3from selenium import webdriver
4
5@pytest.fixture(scope='function', params=['chrome', 'firefox'])
6def driver(request):
7    """Run tests on multiple browsers"""
8    if request.param == 'chrome':
9        driver = webdriver.Chrome()
10    else:
11        driver = webdriver.Firefox()
12    
13    yield driver
14    driver.quit()
15
16# Run tests in parallel
17# pytest -n 4  # 4 parallel workers (requires pytest-xdist)

Best Practices

1. Use Explicit Waits

python
1# ❌ Bad: Hard-coded sleep
2time.sleep(5)
3
4# ✅ Good: Explicit wait
5wait.until(EC.presence_of_element_located((By.ID, 'content')))

2. Unique Locators

python
1# ❌ Bad: Fragile locator
2driver.find_element(By.XPATH, '//div[3]/span[2]/button')
3
4# ✅ Good: Stable locator
5driver.find_element(By.ID, 'submit-button')
6driver.find_element(By.CSS_SELECTOR, '[data-testid="submit-btn"]')

3. Page Object Model

python
1# ❌ Bad: Everything in test
2def test_login():
3    driver.get('https://example.com/login')
4    driver.find_element(By.ID, 'email').send_keys('test@test.com')
5    driver.find_element(By.ID, 'password').send_keys('pass123')
6    driver.find_element(By.ID, 'submit').click()
7
8# ✅ Good: Use Page Objects
9def test_login():
10    login_page = LoginPage(driver)
11    login_page.open().login('test@test.com', 'pass123')

4. Always Clean Up

python
1# ✅ Always use try-finally or context managers
2try:
3    driver = webdriver.Chrome()
4    # Test code
5finally:
6    driver.quit()

Debugging Tips

python
1# Enable verbose logging
2import logging
3logging.basicConfig(level=logging.DEBUG)
4
5# Slow down execution
6from selenium.webdriver.chrome.options import Options
7options = Options()
8options.add_experimental_option('detach', True)
9
10# Run in headed mode (see browser)
11driver = webdriver.Chrome(options=options)
12
13# Add breakpoint
14import pdb; pdb.set_trace()
15
16# Print page source for debugging
17print(driver.page_source)

Conclusion

Selenium with Python provides:

✅ Mature, stable framework ✅ Cross-browser testing ✅ Rich Python ecosystem ✅ Large community support

While newer tools like Playwright offer improvements, Selenium remains a solid choice for browser automation, especially for existing projects and teams familiar with its patterns.

Happy automating! 🚗🐍