Maffo Bulk Upload System for V4 (official release)

Discussion in 'Modules / Plugins / Modifications' started by Maffo, Feb 20, 2008.

  1. Maffo The Master

    To coincide with the official release of 68 classifieds V4 I have adapted my latest mod to run on the v4 system.

    The system has been beta tested and there are no known bugs

    This system also works with v3.1

    Faster than a speeding bullet (hmmm.... maybe not that fast but certainly faster than Ben Johnson in 1988)
    More powerful than a locomotive (I would have to say yes)
    Leaps tall buildings in a single bound (havent tested this one yet)

    This my friends is the Maffo Bulk Upload System.



    What it does:
    Allows you to upload literally thousands of listings to your website via a CSV.file
    A csv file is basically information from a spreadsheet or database saved in a text format.

    This system processes your images, inserts them into your photo folder, resizes them if necessary and also makes the thumbnails for them.

    The most difficult part about writing this system was the knowledge that everybody has a different system with different categories, amount of images,extra fields, listing packages etc. I have done my utmost to make sure, and I am 100%, confident that it will cater for everybodies needs.

    This system has been tested with ZERO server timeouts.

    A benchmark test I ran on a fairly standard, shared server running Apache/linux gave me some really optimistic results.

    5000 listings, each with 2 images of dimensions 800*600px (avg 100kb) set to be reduced to 640*480.

    Thats 10000 images to be processed and also set to a thumbnails

    Each listing also has 2 extra fields, a short description, and a full description

    5000 listings took 76mins 32 secs

    Yes that might seem like a hell of a long time in computer terms but when you we look at the figures we can see that its taking less than a second to..

    * Pull the record from the database
    * xss clean everything!!
    * strip any unallowed html
    * Update your orders table
    * Update your Products Table
    * Add two extra field entries
    * Resize two large images and also create their thumbnails


    Strip out the images and the system can fly like a knife through butter.

    I have also written a delete feature which allows me to remove everything from that last import including the images from the image folders. This is incase I upload my csv only afterwards to realise I have placed something wrong. (even though there are lots of checks before the actual processing of the csv, I felt this was a necessary feature)

    This benchmarked at under 3mins which is pretty damn good

    I then took the same csv sheet, removed the 10000 images and submitted 5000 plain text ads to the upload system. This also came in at under 3mins so I clearly tell that is the image manipulation that slows this whole process down.

    There is a category validator which compares what is on your CSV to what is allowed in your db (works like a dream and saves you having to manually type out any categories)

    There is an Image validator which checks that the images in your CSV file actually exist and if not allows you to rectify things before continuing.

    Prepping the information can be the most time consuming so I have made it possible to back up your work at any time so you can continue at a further time.

    Like I said earlier this is by far the finest work I have done so far, and has pushed new limits for 68 users.

    For those with property websites, there is now the ability to contact hundreds of estate agents and get them to send you their properties, the same with cars, boats whatever?

    On testing, it seemed strange seeing my empty site suddenly with 5000 ads?? and best of all, how long would it have taken me to put them all in manually??? I dread to think, at lets say 2 mins an ad? Thats a whole lotta man hours!! 166 to be precise and thats without a coffee or cigarette break!!
    Any questions just fire away. Apart from how much it costs. I really dont know yet but it isnt going to be peanuts. I really feel this has its value.

    Maffo Bulk Uploads Readme

    These are two reviews I have received from clients who have used the system on live sites.

    If you require the mod please contact me via Private Message

    Any questions, please post on this thread
  2. manilaboy Customer

    Maffo's Banner Mod

    Hi Maffo,

    Will your banner mod work on the newly released V4?
  3. Maffo The Master

    No Sorry Mike, and I have no plans to re-write the banner mod for v4. For a start there is no membership system which the banner mod relies on.
  4. bearcat Customer

    Interesting.

    This would seem to be fairly easy on the advertiser's part, except (possibly) for one thing.

    Is there a simple way of copy and pasting picture names onto a csv? It seems like there should be, but I can't think of one. Is there a feature in Office, etc., that allows for something like this?

    Many picture names will look something like DSC11028.jpg. If there are 200 ads, then typing out, say, 5 of these per ad, or 1000 in total would not be fun.
  5. Maffo The Master

    That sounds like it would be a real pain Bearcat and Id hate for you to be sat at your machine for an hour.

    Its for this very reason I have included the option to 'generate image name from title'. Its not the perfect solution but what it does, is as it says it takes the main title of the ad, such as 'Lawnmower for Sale' and names the images 'lawnmower for sale 1' 'lawnmower for sale 2'. The actual physical name of the image doesnt change but the title etc will when the user is looking at the viewlisting.php page
  6. bearcat Customer

    I'm not sure if I follow. Since it does not change the actual names of the images themselves, how would it bypass this problem?

    Great module, btw! I expect to be purchasing it after I transition to v4.
  7. Maffo The Master

    The image name ie the file name DC00123.JPG will remain as it is. There is no need to change it.

    However if you look at the viewlisting.php each image has a title for example 'car front' 'car back'. This is all the user ever sees. They dont see the filename of the image.

    My mod has 3 options for images. You can....

    A: For each image, image 1, image 2, image 3, you can add a column, image1 title, image2 title etc so you can manually place these inside your csv. The bulk upload system will then save the image title from these columns.

    B: The system can use the filename to create an image title. EG front_of_car.jpg, back_of_car.jpg, if these are your image files, the filename will be stored as the image title.

    C: Generate image titles from the overall title of the listing. EG 'used car for sale'. Lets say your client has four images, the images will appear as 'used car for sale 1' 'used car for sale 2'

    I have written a fully comprehensive guide for this here..

    BULK UPLOAD SYSTEM USAGE GUIDE

    It looks a little daunting but once you have done it once or twice, its very very easy.

Share This Page