RSS site sync.

[Lhotch]

Ive spent a ton of time working on a core scrip to handle the synchronizing of ads on a 68C V3 site with an RSS feed and am looking for inpiut on those of you who may have a need or desire for this type of functionality.

The reason I call it a "sync" as opposed to an import is because, while the script does do an initial import of ads the goal behind it is to synchronize the ads on the 68C site with a live RSS feed.

The primary target for this is 68C sites that want to carry ads for dealers or brokers where the dealer/broker is the primary contact for the item being sold.

Since all ads need to be associated with a registered user the goal is to have the dealer/broker register on your 68C site (or admin can do it). Then a cron job is created on the 68C web server to run the sync script once a day, or how ever often is desired and process the feed......

• The script reads the rss feed
• it compares ads in the feed with the ads currently on the 68c site
• if an ad in the feed already exists on the 68c site it is ignored.
• if an ad in the feed does not exist on the 68c site it is added under the listing broker/dealer.
• if there is an ad on the 68c site that is no longer in the rss feed, its assumed no longer available and emoved from the 68c site.
• Images in the feed will be downloaded, resized accordingly and have thumbs automatially created.

Due to all of the variables in in site layouts like categories and extra fields logic to handle the sorting of inbound ads and how to handle variances in the extra fields I manually look at the source feed and destination site and along with 68c site owner feedback we plan out how to handle categories and extra fields and this information is hard coded into the script.

So, long story short, im curious how many would find something like this usefull and if you would find it usefull would you prefere a front end where you personally map the fields in the feed to your fields or would you rather leave it to me to implement these changes on a feed by feed basis and supply the script so you didnt have to mess with it?

[Kelly Olsen]

Hi Larry,

I thought I'd give you my input on this..

I would be very interested in this feature. In my situation, I would be OK with you setting each feed up, but I will deal with fairly low numbers of dealers as I'm in a very niche market.

I have a dealer that would be interested in signing up, but they want us to pull the data from their website and "sync" it with our site. Actually I talked to you quite a while ago about this, but I can't remember what the solution was then. As I recall, they have a database driven site that uploads a "flat file" to a web server, and we'd have to go in and grab the data from the flat file to compare to, and update, our database. How difficult would it be for them set up an RSS feed to work with your script (I think they have a pretty good IT guy)?

Thanks for all your great work in 2007, I look forward to 2008 (and V4)!

HAPPY NEW YEAR!

Kelly

[Lhotch]

They wouldnt need to create an RSS feed, we could work with a web visible flat file as long as its layout is consistant. Assuming it is a CSV (comma seperated value) datafile we can work with that.

What really gets tricky is mapping their category names. This is generally pretty straght forward if the ads are on another classified site where there is some form of referential integrity employed bu it can get real ugly, real fast if there is no referential integrity employed and therefor no set category names at the data source.

What I mean by referential integrity is that, on a 68c site for example, when an ad is placed you have to select the category from a predefined list and therefor, aside from when a category may be added or removed, there is a consistant list of category names that is to be expected. I have used my import script with a source site that used blank text fields for the category. Even though ads were handled by employees of the site there was nothing preventing them from mispelling a category and thus when the logic of my script looked at the category of the ad to import it there was room for human error and therefor the possibility it may not match and of the mapped target fields. In cases like this we need to decide how to handle it. We can ignore it, or dump it in a catch all category and the site admin (ie you) can check every couple days and simply manually edit the ads and change their category.

Happy New Year to you as well, and thanks for the kind words.