How to Use Google Indexing API to Submit URLs in Bulk using Python?

This guide is a non-programmer’s guide to using the Google Developers Indexing API to update or remove pages from the Google index instantly using Python. The objective of this guide is to use a python script to submit URLs in bulk to Google for crawling to fix some of your page indexing issues related to indexable URLs being crawled but currently not indexed. Follow the step-by-step guide illustrated with images to use Google Indexing API with Python.

Step-by-Step Guide to Using Google Indexing API through Python

Step 1: Install Python

Head to https://www.python.org/ to download and install the latest Python version. For the purpose of this guide, I would be installing the Windows version since I use a PC. But the process is similar for Mac. After the executable is downloaded, follow the self-explanatory installation to complete the installation.

Install Python

Step 2: Setting up the Indexing API key in your Google Developer Console

Head to Google Developer Console. Google Indexing API is free. So do not worry about adding a billing account if you are just using the Indexing API.

Create new project on Google Cloud
  • Create a New Project. Give it a name that is relevant and easy to remember. Since I am going to be using this for sorting indexing issues for Flannels, I’ll name the project ‘Flannels Indexing API’. Click on the Create Button.
Create New Project Google Indexing API Google Cloud
  • Head to the Newly Created Project (Flannels Indexing API project). After the new project is created, keep it as the selected project as shown in the screenshot below and follow the next steps.
Navigate to New Project on Google Cloud
  • Setup APIs & Services Credentials. For this, you need to set up the API credentials by visiting the Credentials link under the APIs & Services menu using the left sidebar. Once you are on the Credentials window, click on ‘Manage Service Accounts’ and create a new Service Account.
Setup APIs & Services Credentials
Manage Service Accounts Google APIs Cloud
Create Service Accounts Google APIs Cloud
  • Create a Service Account. On the Create service account screen, you can follow the steps shown on the screen to set up your new service account. Please name the account with a memorable name that is relatable to your project. I am just going by the name ‘Flannels Indexing Service Account’ since this project is meant for Flannels. Once you add the name and click on create and continue, you will need to select a role to grant your service account access to the project. Please select the ‘Owner’ role and click the done button.
Create Service Accounts on Google APIs Google Cloud
Create Service Accounts on Google Cloud APIs
  • Once you are done with the above step, you will be taken to the ‘Service accounts for the project screen as shown in the screenshot below. You will notice an email address (gservicesaccount.com email address) associated with your project and service account. Make a note of this email address as you will be using this email address in Step 3 of this guide. So, my advice is to save the email id onto a local notepad file.
Service Accounts for Project Google Cloud Screen
Manage Keys Service Accounts for Project Screen
  • Create a Key for your Service Account. Click on the three dots under the actions column and select the ‘manage keys’ option. You will be taken to the ‘Keys’ screen where you can add a key by clicking on the ‘Add Key’ button. Keep the JSON option selected and click on the ‘Create’ button. A small JSON file will be downloaded. This is an authentication file which will verify your API access. A window displaying ‘Private key saved to your computer’ will appear which can be safely closed.
Add Key Service Accounts for Project Google Cloud Screen
  • Enable API. The next step is to enable the API. To do this, click on the ‘Library’ link under the API & Services menu section. Search for ‘indexing API’ and select the first Indexing API option from the two options shown. Click the ‘enable’ button on the next screen.
Enable APIs Google Cloud Screen
Enable Indexing API Screen Google Cloud

This completes all the steps of ‘Setting up the Indexing API key in your Google Developer Console’.

Step 3: Grant Owner Access to GServicesAccount Email in your Google Developer Console

In this step, you will grant owner-level access to the gservicesaccount.com email address that was created in the previous step. If you did not save the email address in the previous step, you can view the email address by selecting the Credentials link on the left sidebar. Once you have the email address copied, visit your Google Search Console account and select the website property that you want to use the indexing API. I have used it for Flannels as shown in the screenshot. You will need to be an owner of the property yourself in order to grant new ownership access to the gserviceaccount.com email.

Google Cloud Credentials Screen
Google Search Console Owner Access APIs Google Services Email Account

Step 4: Final Step – Running the Python Script

So far, in the previous steps, you completed the set-up of the Indexing API key in your Google Developer Console, installed Python and granted owner access to the API key in your search console. Now we are in the final step where we will use the Google indexing API with python script.

  • Install Some Libraries. Go to your Windows command prompt (search for CMD if you are using Windows or terminal if you are using a MAC). We will need to install some libraries before being able to use the indexing API. Run the following two commands on your command prompt. The httplib2 and the pandas’ library will download and install in your system.

For Windows Users

Command 1: pip install oauth2client httplib2

Command 2: pip install pandas

For Mac Users

Command 1: pip3 install oauth2client httplib2

Command 2: pip3 install pandas

Install Python Libraries Google Indexing API
  • Create a folder to include 3 files – Python Script, URLs.csv and the JSON file that was previously downloaded. The JSON file that was downloaded in Step 2 will need to be moved into this folder. Download the urls.csv which will contain the list of URLs you want to submit to Google and the indexing.py python code using this link and move them all to one folder like the screenshot below. Full credit to the programmer for sharing the code here.
  • Update the Python Code and the CSV File. The only change you need is to make a small change in line 8 of the indexing.py python code. Change the name of the JSON file on this line JSON_KEY_FILE = "YOUR_JSON_FILE.json" with the name of your JSON file. You can edit the file using software like Textpad or Notepad++. You can use Brackets on Mac. Next, you will need to add the URLs in column A of the urls.csv file. Do not remove the first row containing URL and Date values. You can add a maximum of 1000 URLs at a time.
URLs CSV for Google Indexing API Project
Python Script Google Indexing API
  • Execute the Script. Open the command prompt (on your PC) or terminal (if using a Mac) and navigate to the folder containing the 3 files. You can use the change directory CD command as shown in the screenshot below. Once you are in the right folder, you can execute the script using the below command.

For Windows Users

Command: python indexing.py

For Mac Users

Command: python3 indexing.py

Google Indexing API Python Script Execution Command Line

Once you enter the command and press the enter button, you will see Google URL update notifications for all the URLs that were entered in your urls.csv one by one along with timestamps.

This concludes the guide to Google Indexing API implementation through Python on the command line. Please note that this is not a guaranteed way of indexing your URLs. It is the same as fetching and submitting URLs on the search console. The only advantage of this method is you can do this in bulk to save time.

0 comments… add one

Leave a Reply

Your email address will not be published. Required fields are marked *