Maffo Bulk Upload System for V4 (official release)

[Maffo]

To coincide with the official release of 68 classifieds V4 I have adapted my latest mod to run on the v4 system.

The system has been beta tested and there are no known bugs

This system also works with v3.1

Faster than a speeding bullet (hmmm.... maybe not that fast but certainly faster than Ben Johnson in 1988)
More powerful than a locomotive (I would have to say yes)
Leaps tall buildings in a single bound (havent tested this one yet)

This my friends is the Maffo Bulk Upload System.



What it does:
Allows you to upload literally thousands of listings to your website via a CSV.file
A csv file is basically information from a spreadsheet or database saved in a text format.

This system processes your images, inserts them into your photo folder, resizes them if necessary and also makes the thumbnails for them.

The most difficult part about writing this system was the knowledge that everybody has a different system with different categories, amount of images,extra fields, listing packages etc. I have done my utmost to make sure, and I am 100%, confident that it will cater for everybodies needs.

This system has been tested with ZERO server timeouts.

A benchmark test I ran on a fairly standard, shared server running Apache/linux gave me some really optimistic results.

5000 listings, each with 2 images of dimensions 800*600px (avg 100kb) set to be reduced to 640*480.

Thats 10000 images to be processed and also set to a thumbnails

Each listing also has 2 extra fields, a short description, and a full description

5000 listings took 76mins 32 secs

Yes that might seem like a hell of a long time in computer terms but when you we look at the figures we can see that its taking less than a second to..

* Pull the record from the database
* xss clean everything!!
* strip any unallowed html
* Update your orders table
* Update your Products Table
* Add two extra field entries
* Resize two large images and also create their thumbnails


Strip out the images and the system can fly like a knife through butter.

I have also written a delete feature which allows me to remove everything from that last import including the images from the image folders. This is incase I upload my csv only afterwards to realise I have placed something wrong. (even though there are lots of checks before the actual processing of the csv, I felt this was a necessary feature)

This benchmarked at under 3mins which is pretty damn good

I then took the same csv sheet, removed the 10000 images and submitted 5000 plain text ads to the upload system. This also came in at under 3mins so I clearly tell that is the image manipulation that slows this whole process down.

There is a category validator which compares what is on your CSV to what is allowed in your db (works like a dream and saves you having to manually type out any categories)

There is an Image validator which checks that the images in your CSV file actually exist and if not allows you to rectify things before continuing.

Prepping the information can be the most time consuming so I have made it possible to back up your work at any time so you can continue at a further time.

Like I said earlier this is by far the finest work I have done so far, and has pushed new limits for 68 users.

For those with property websites, there is now the ability to contact hundreds of estate agents and get them to send you their properties, the same with cars, boats whatever?

On testing, it seemed strange seeing my empty site suddenly with 5000 ads?? and best of all, how long would it have taken me to put them all in manually??? I dread to think, at lets say 2 mins an ad? Thats a whole lotta man hours!! 166 to be precise and thats without a coffee or cigarette break!!
Any questions just fire away. Apart from how much it costs. I really dont know yet but it isnt going to be peanuts. I really feel this has its value.

Maffo Bulk Uploads Readme

These are two reviews I have received from clients who have used the system on live sites.

Just thought Id share my thoughts on the Bulk Upload System Maffo developed recently.

What does it do ?

-It allows you to upload ads to your site in bulk. dozens/hundreds/thousands at a time.

The actual task of doing the upload is quite simple and intuitive, the hard part is collecting the data and images from the advertiser, particularly images.

This is how I do it....

-I send my client a spreadsheet which contains the columns/fileds required to place an Ad eg: Category/Ad Title/Description/Price/Extra Fields/image name/etc

-They send it back, along with all the images named as they are in the spreedsheet.

-I check that all the images are present and run them through a image resiing program which takes seconds and that all the data is in the correct places.

- Then I upload the images to the server, save the spreedsheet as a .csv file

- Login to Admin/Modules/Maffos Bulk Upload System, point it to my .csv file

-Then all you have to do is follow the prompts, along the way it checks the quality and layout of the data by showing you a few random ads as they will look when uploaded and it lets you know if an image is missing.

- Once the upload process is finished(which does not take long, depending on how many ads there are) you are given one chance to "Undo" it if something is not right.

I have used this on a live site and it works great.

By far and away the most time consuming and hardest part is collecting the data from the advertiser in the right format in the spreedsheet then checking it.

I would highly recommend this module to anyone who has numerous ads to place. To place large amounts of ads manually would take a LOT longer and would be very tedious work.

I would be happy to answer any questions anyone has from a users point of view.

Job well done Maffo!

Yes, this is now available for V4. I have been running it on a live V4 RC10 site for about 3 weeks now, and it works great for me. Took only a few minutes to process a batch of 400+ listings with pictures. Lots of safety features, including the option to wipe out the entire upload in case something goes wrong. (Which it shouldn't, since you are walked through each step and alerted to fix any errors before the upload is processed.)

If you require the mod please contact me via Private Message

Any questions, please post on this thread

[manilaboy]

Hi Maffo,

Will your banner mod work on the newly released V4?

[Maffo]

No Sorry Mike, and I have no plans to re-write the banner mod for v4. For a start there is no membership system which the banner mod relies on.