Have Geek Brain

Spider Generation Using Scrapy, Spider Generation Error

I just downloaded Scrapy (web crawler) on Windows 32 and just created a new project folder using scrapy -ctl.py startproject dmoz command in dos. Then I proceeded to create the first spider using the command:

scrapy-ctl.py genspider myspider myspdier-domain.com

but it doesn't work and returns an error:

Error running: scrapy-ctl.py genspider, Cannot find project settings module in python path: scrapy_settings.

I know I have the correct path (to python26 / scripts), but I am having a hard time figuring out what the problem is. I'm new to both scrapy and python, so there's a good chance I did something important.

Also, I am using eclipse with the Pydev plugin to edit the code in case that might cause some problems.

+2


a source to share


3 answers


There is a difference between PATH and PYTHON_PATH. Is PYTHON_PATH set correctly? This way is what packages / modules are supposed to include in python.



+2


a source


use scrapy-ctl.py in your project directory. what the script will know about these project settings. the main scrapy-ctl.py has no idea about these specific project settings.



+1


a source


Set the environment variable PYTHONPATH in python26 / scripts.

0


a source






More articles:


All Articles