An Automated Craigslist Search Script
This is a set of scripts that will automatically search the Craigslist lists for specified terms. Any listings the Python script finds that match it stores in a SQLite database. A PHP script displays the database.
Features
- Can search any number of listings for any number of terms.
- Automatically deletes listings that have been removed.
- Marks new listings since your last visit.
- Displays the date the listing was found and the time of the last update.
Files
| File | Last modified | Size |
|---|---|---|
| craigslist.php | 2011-06-20 17:09 | 2Kb |
| craigslist.py | 2011-06-20 17:09 | 3Kb |
| craigslist.sql | 2010-06-07 16:23 | 241b |
Instructions
- Create a SQLite database for the scripts to use.
- Edit the scripts and fill in the SQLite database name.
- Create the 'listings' table using the provided SQL code.
- Set up a cron job to run the Python script to update the database.
- Edit the Python script to search the lists and terms that you want.
Caveats
Both of these scripts were put together in a few hours so they're pretty ugly with no documentation but the code is fairly simple. This is only the second or third time I've used Python and it's been many years since I touched PHP so they're probably both a little rough around the edges but they work. You will need to provide the correct URL for the listing you want to pull in the Python code. Also if you want it to search different parameters you'll need to add a for loop to make that work.
You should have a basic understanding of Python and PHP to use these scripts as they will probably need a bit of coaxing to get to work. I'm sorry they're not easier to set up but if I get enough interest I'll put some time into making them more user-friendly.
I recently (June 2011) converted these scripts to use SQLite instead of MySQL. The SQL code linked above will not work with SQLite and I just haven't had time to update it. Therefore you will have to hack at it a little to get things to work. The table structure is the same but SQLite uses some different parameters than MySQL for the primary key IIRC. You will also have to give your web server write permission to the SQLite database.
License
Copyright © 2011, Mike Roddewig.
All rights reserved.
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License v3 as published by the Free Software Foundation.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>.