python - Scrapyd init error when running scrapy spider -
i'm trying deploy crawler 4 spiders. 1 of spiders uses xmlfeedspider , runs fine shell , scrapyd, others use basespider , give error when run in scrapyd, run fine shell
typeerror: init() got unexpected keyword argument '_job'
from i've read points problem init function in spiders, cannot seem solve problem. don't need init function , if remove still error!
my spider looks this
from scrapy import log scrapy.spider import basespider scrapy.selector import xmlxpathselector betfeeds_master.items import odds # parameters myglobal = 39 class homespider(basespider): name = "home" #con = none allowed_domains = ["www.myhome.com"] start_urls = [ "http://www.myhome.com/oddxml.aspx?lang=en&subscriber=mysubscriber", ] def parse(self, response): items = [] tracecompetition = "" xxs = xmlxpathselector(response) oddsobjects = xxs.select("//oo[oddstype='3w' , sport='football']") oddsobject in oddsobjects: item = odds() item['competition'] = ''.join(oddsobject.select('tournament/text()').extract()) if tracecompetition != item['competition']: log.msg('processing %s' % (item['competition'])) #print item['competition'] tracecompetition = item['competition'] item['matchdate'] = ''.join(oddsobject.select('date/text()').extract()) item['hometeam'] = ''.join(oddsobject.select('oddsdata/hometeam/text()').extract()) item['awayteam'] = ''.join(oddsobject.select('oddsdata/awayteam/text()').extract()) item['lastupdated'] = '' item['bookie'] = myglobal item['home'] = ''.join(oddsobject.select('oddsdata/homeodds/text()').extract()) item['draw'] = ''.join(oddsobject.select('oddsdata/drawodds/text()').extract()) item['away'] = ''.join(oddsobject.select('oddsdata/awayodds/text()').extract()) items.append(item) return items
i can put use init function in spider, same error.
def __init__(self, *args, **kwargs): super(homespider, self).__init__(*args, **kwargs) pass
why happening , how solve it?
the answer given alecx :
my init function :
def __init__(self, domain_name):
in order work within egg scrapyd, should :
def __init__(self, domain_name, **kwargs):
considering pass domain_name mandatory argument
Comments
Post a Comment