How To Use Python For SEO Redirect Mapping

Website migrations often lead to broken links and ranking losses if redirects are not mapped correctly. Python makes this process faster and more accurate by automating URL matching using page titles, headings and fuzzy matching.

Instead of manually mapping hundreds or thousands of URLs, you can use Python to compare old and new site crawl data and generate redirect recommendations in minutes. This is especially useful during SEO website migrations, where preserving rankings, traffic and user experience is critical.

Why Use Python for Redirect Mapping?

Manual redirect mapping becomes difficult at scale and increases the risk of:

• Missed URLs
• Incorrect redirects
• Redirect chains
• Lost SEO equity
• 404 errors after launch

Python helps automate the process by:

• Comparing old and new URLs
• Matching pages using titles and headings
• Scoring similarity between pages
• Exporting redirect recommendations for review

For larger projects, this sits naturally alongside broader technical SEO work, including crawl optimisation, indexation checks and post-launch monitoring.

What You’ll Need

• The latest version of Python
• VS Code (or alternative code editor)
• Screaming Frog (or alternative website crawler)

How To Install Python

Download the latest version of Python from:

Python.org

During installation:

• Tick “Add Python to PATH”
• Click Install Now

Once installed, restart your terminal or command prompt.

Check that Python is installed correctly

Open Terminal (Mac) or Command Prompt (Windows) and run:

python --version

You should see something similar to:

Python 3.12.3

If that does not work on Windows, try:

py --version

Create a Project Folder

Create a new folder somewhere easy to access, for example:

seo-redirect-mapping

This will be the folder you will place your Screaming Frog crawls in.

Open the Folder in VS Code

Download:

Visual Studio Code

Then:

• Open VS Code
• Click File → Open Folder
• Select your project folder

Create a new Python file called:

redirect-mapping.py

You are now ready to install the required libraries and run the script.

Gather Website Crawls

Before starting, gather crawl exports from both the old and new websites using a crawler such as Screaming Frog.

Download:

Screaming Frog

Your exports should include:

• URL
• Page title
• H1
• H2

Save both crawl exports as .xlsx files.

Preliminary Steps

Before running the steps below, open the integrated terminal in VS Code by clicking Terminal → New Terminal from the top menu.

This opens a terminal window directly inside your project folder. The pip install command in Step 1 is entered here.

For Steps 2 to 8, paste all of the code blocks into your redirect-mapping.py file in order, then run the script from the terminal using python redirect-mapping.py.

Before installing any libraries, it is recommended to set up a virtual environment. This keeps your project dependencies isolated and avoids conflicts with other Python projects on your machine.

Virtual environment — Mac

python -m venv venv
source venv/bin/activate

Virtual environment — Windows

python -m venv venv
venv\Scripts\activate

Once activated, you will see the environment name appear in your terminal. All libraries installed with pip will now be contained within this project folder. To deactivate the environment when you are finished, simply run deactivate as below:

deactivate

Make sure the virtual environment is active in your terminal before moving on to Step 1. You will see (venv) at the start of your terminal prompt when it is activated. It should look a little something like this:

(venv) your-machine-name:seo-redirect-mapping yourname$

Step-by-Step Guide

1. Install Required Libraries

pip install pandas polyfuzz rapidfuzz openpyxl

rapidfuzz is required separately for the matching model import to work correctly.

2. Import Libraries

import pandas as pd
from polyfuzz import PolyFuzz
from polyfuzz.models import RapidFuzz

3. Load Crawl Data

old_urls = pd.read_excel("old_site_crawl.xlsx")
new_urls = pd.read_excel("new_site_crawl.xlsx")

old_urls = old_urls.rename(columns={'Address': 'Old URL', 'Title 1': 'Old Title'})
new_urls = new_urls.rename(columns={'Address': 'New URL', 'Title 1': 'New Title'})

This assumes your Screaming Frog crawl exports use the default column names Address and Title 1, which are renamed in the code for clarity:

• Title 1 → renamed to Old Title / New Title
• Address → renamed to Old URL / New URL

4. Prepare Title Data

old_titles_df = old_urls[['Old URL', 'Old Title']].dropna()
new_titles_df = new_urls[['New URL', 'New Title']].dropna()

old_titles = old_titles_df['Old Title'].tolist()
new_titles = new_titles_df['New Title'].tolist()

5. Run Fuzzy Matching

matcher = PolyFuzz(RapidFuzz())
matcher.match(old_titles, new_titles)

results = matcher.get_matches()["RapidFuzz"]

The output includes:

• Original title
• Suggested matching title
• Similarity score

6. Filter High-Confidence Matches

filtered_results = results[results["Similarity"] >= 0.90]

You can lower this threshold for larger or messier migrations, but anything below 0.80 should usually be reviewed manually.

7. Map Matches Back to URLs

redirect_map = filtered_results.merge(
    old_urls[['Old URL', 'Old Title']],
    left_on='From',
    right_on='Old Title'
)

redirect_map = redirect_map.merge(
    new_urls[['New URL', 'New Title']],
    left_on='To',
    right_on='New Title'
)

final_redirects = redirect_map[
    ['Old URL', 'New URL', 'Similarity']
].copy()

final_redirects.columns = [
    'Old URL',
    'New URL',
    'Confidence Score'
]

8. Export the Redirect Map

final_redirects.to_excel(
    "redirect_mapping.xlsx",
    index=False
)

You will now have an Excel file containing:

• Old URL
• Suggested new URL
• Match confidence score

Review lower-confidence matches manually before implementation.

Tips for Better Redirect Matching

To improve accuracy:

• Match using H1s if titles are inconsistent
• Remove duplicate titles before matching
• Exclude paginated or parameter URLs
• Manually override important pages
• Review all low-confidence matches

For important migrations, it is also worth carrying out an SEO audit before launch so technical issues, content gaps and redirect risks are identified early.

Final Thoughts

Python dramatically simplifies redirect mapping during SEO migrations. By automating title and content matching, you can generate scalable redirect recommendations quickly while reducing manual errors.

The workflow above gives you a reliable foundation that works well for most migrations and can easily be expanded for larger, more complex websites.