Append URL's in a given substring from a range of integers - scrapy

Question

I am trying to format the URL's of a given file.
The URL's have this format:
http://randomtext&of=randomtext&=.com
There is an init method that reads the url's from a file:
def __init__(self,filename=None):
if filename:
with open('urls.txt') as f:
self.start_urls = [url.strip() for url in f.readlines()]
I can append on the end of the url with a range of integers:
def __init__(self,filename=None):
if filename:
with open('urls.txt') as f:
self.start_urls = [url.strip() for url in f.readlines()]
self.start_urls = [url + str(i*10) for i in range(0,25) for url in self.start_urls]
My question is how to append the string on the substring 'of=' on the above URL's with a number of integers eg:
http://randomtext&of=1randomtext
http://randomtext&of=2randomtext
http://randomtext&of=3randomtext
Thank you in advance!

Answer 1

You could use the sub() function from pythons re-module to replace the numbers in the urls with your numbers from a given range.
So build upon your provided example, one possible solution could be this:
import re
def __init__(self,filename=None):
if filename:
with open('urls.txt') as f:
self.start_urls = [url.strip() for url in f.readlines()]
self.start_urls = [re.sub("[0-9][0-9]*", str(i*10), url) for i in range(0,25) for url in self.start_urls]
You may want to check if there are other numbers in your urls that you don't want to be substituted and change the regex accordingly.