Magento: Working on a Product Import Script

Project Cancelled

Due to lack of resources (financial, time, and help), I have to officially cancel this project. This is just too much of an undertaking for me alone. Sorry to disappoint…

While Magento did a good thing of creating the ability to import a CSV from within the Magento admin, it still lacks a bit, and it is extremely slow. It has improved in speed since 1.0 came out (it took me 15 hours to import 4500 products), but it still is very slow. It can even be a little overly picky sometimes.

That is why I started learning the eav database setup early, and began importing my own products with my own PHP script. With the first script I made, I was able to import those same 4500 products in 18 seconds instead of 15 hours. That is a time savings of 14 hours, 59 minutes, and 42 seconds.

My script wasn’t at all built smart though. It required a very specific number of columns, in a specific order, and with a specific text delimiter. All of the attribute ids and other variables were all hard-coded into the script, making it tedious to customize for a different system. I just recently learned that the attribute id values for different installations of Magento can be completely different.

So, I am on a mission to create a “smart” product import script. I plan to have different configuration options all at the top of the php script, which would keep the user out of the core and make setting it up easy. My goal is to make it so that you can create a CSV file of your products, with headers that match Magento’s attribute_code in any order you want it in. Even custom attributes that you create will be easily imported.

The script has already been started. Much of the core import features are already done. I took what I already had and gave the script to a friend of mine to test out, and he imported 12,630 products in 60 seconds. But, that script still required a specific column order.

Current Features

  • Imports products – FAST!
  • Ability to see the data of your CSV displayed in a table
  • Automatic attribute_id lookup
  • Includes Mage.php class to create url_key and url_path
  • Ability to set default values


  • Updates product if it already exists
  • Accepts any column order
  • Accepts custom attribute values
  • Importing images
  • Importing products into categories
  • Importing tier pricing
  • Graphical interface
  • Created as a Magento module

I’ll keep you updated. If you have any interest in this project, please let me know – post a comment. Also, if there are any features or ideas you would like to suggest, please let me know too!

This entry was posted in Magento. Bookmark the permalink.

42 Responses to Magento: Working on a Product Import Script

  1. Kevin,

    We have just finished developing an order import program that uses Magento’s API to create the orders in CounterPoint SQL v8.3.9 and then change the status in Magento. You can read more about it here and feel free to contact me through the site if you have any interest.

  2. Kevin says:

    Hey Josh.

    I assume that you’ve stopped the development of the import script … is there any chance that one might get to use your script and maybe change it to fit my own needs ?

    that would save me a lot of time and I would greatly appreciate it !

    please let me know !


    • Josh Pratt says:

      Kevin – I stopped a LONG time ago. I don’t think I even have the code anymore. Looking back now, it was such a piece of junk anyway, you really wouldn’t want it.

  3. Tim Redman says:

    Josh, you mentioned that you started doing some work integrating Magento with CounterPoint SQL. How far did you get on that? Do you have any documentation left over from the project? We’re trying to synchronize inventory between Magento and CounterPoint, and have only really had success pulling data from CP, not adjusting back into it. Any pointers you have would be appreciated.

  4. Krishna says:

    hi Josh,

    I am a Drupal Developer and new to magneto ,I need to get import orders and complete customer information from database on daily basis without logging in admin panel,i need a script which will run daily through a cron, and will import new orders and customer information daily and generate it in a XML file,any help would be greatly appreciated.


    • Josh Pratt says:

      Krishna, I bailed on the import script a long time ago. For one, I wasn’t getting any support from the community besides “Are you done yet!?”, and for two, the API is really good enough for all my uses. I would suggest utilizing the API.

  5. websafe says:

    Hello! Sounds nice, but You have no time… Can You please just share the method You used to update import products in 18 seconds? Using $mageProduct = Mage::getModel(‘catalog/product’); …. takes loooooong. I have about 10000 products, it takes HOURS or even DAYS…. :-(

    • Josh Pratt says:


      Sorry, but I stopped development of this long ago. There are just too many challenges, and no good support from the community. The method I used just wrote directly to the database, bypassing all Magento code. It was dirty and not flexible. I don’t even think I have that code anymore.

  6. Manoj Ninave says:

    helo sir that is very goood post. could u please give me this script.God bless you.take care

  7. Luis Abarca says:

    Great script, im very interested.

    Esta buenisimo, estoy muy interesado.

  8. Lee says:

    I have a client who is wanting to set up a solution to show over 17,000 products, all currently in an excel spreadsheet with a field for ‘image name’.
    would this ‘import script’ (when complete) be able to help me?
    So far, Magento seems the best product, unless anybody can suggest anything else?

  9. Chris says:

    I am interested in helping with the import script. I am working on a project where large amounts of products have to be imported on a regular basis, so Magento’s import isn’t really an option. I have gone through the script today and cleaned up some bugs, added support for categories and images and some other things. I think having a more robust script would be useful, so I can help out while my hands are dirty anyway. Does anyone know why Magento’s import is so ridiculously slow?

  10. Jake Rutter says:

    Great script, I think it will be a good alternative to the Magento Import tool which just cannot seem to get the job done in a timely manner.

  11. Josh Pratt says:

    Hi everyone,

    Due to the business of work – and not having anyone solid to work on this project with – the project isn’t going any further at this point. It’s a very, very difficult task to create a product importer that is solid and very easy to use. If I can find someone that is willing collaborate with me then I would be willing to further the project – but right now, I just don’t have the resources to do it…

  12. Scott says:

    How goes it? We would be very interested in this script and donating to the cause. You could save us a substantial amount of time!

  13. smith says:


    Great news. thanks for your update

    can you release this script. i have very bad requirement for you great script

    thanks again

  14. Josh Pratt says:


    Sorry Gilad. The script is pretty much as-is for now. Sounds like the problem isn’t with my script though if the error is telling you something is wrong on line 73 of a Mage file.

  15. Josh Pratt says:


    I finally got someone who is interested in helping me. We are re-looking at the whole project. Not sure what we are going to do yet, or how long it will take.

  16. GiladG says:

    I keep getting an error that MySQLi.php cannot be included. It results in /app/code/core/Mage/Core/functions.php on line 73.

    It seems to build the include path very screwed up.

    Anyone has an idea?

  17. Jonath says:

    Hye !

    Any news from your script ?

    I’ll be very happy to donate too !

    When it’ll be released ?


  18. William says:

    Very interested in this, make sure you place a donation button just so we can say thanks.

    Will it have the ability to import custom options?

    Good job, thanks

  19. Mike says:

    Hi Josh,

    Great work so far with the script, I was wondering if I could offer some help with regards to the OO php you have been talking about on your blog. We were looking to create a similar import script when I came across yours, and you seem to be tackling the major features.

    We were wondering why importing products with custom attributes is not on the 2.0 release – it is going to be a necessity for importing our catalog and I’m sure the same could be said for many others.

    Could you drop me an email – I have been unable to get in contact with you through irc or find a contact address for you.


  20. Massimo says:

    You are my hero! Please let me have a look even if it’s not the final release.

  21. Office Saver says:

    I would definitely pay for a script that did all this… I honestly can’t wait for the final release of this.

    Thanks for your dedicated work in improving magento, as a professional e-commerce platform.


  22. websuperman says:

    I am not getting any errors. I did comment out the custom attribute imports because that was throwing errors (commented out with leading // ) but now I am not getting errors and the part is imported, just missing a lot of info. Feel free to email me to keep down the clutter here if you’d like.

  23. Josh Pratt says:


    Well, it is fairly straight-forward, especially sinces it’s almost totally procedural. Are you getting errors?

  24. websuperman says:

    I am very interested in the upcoming script. At the moment I am trying to make the current import script work with no luck. It imports the ID, Type, Attribute Set Name, SKU & qty but nothing else. Using 1.2.1, any help or pointers you could provide would be appreciated.

  25. Josh Pratt says:

    While financial help is *nice*, none needed. Money is not setting me back from completing it – time is. I have a full-time job, a family, and plenty of other things I’m involved in.

    I’d be happy to see your script when it is done though!

  26. mitch says:

    I could offer financial help. Currently I want to develop an import script just like the one we used for osCommerce where we have a column in the CSV for “category” instead of category_id’s

    And what I liked best about the osCommerce import was that if your category didn’t exist and you added it to the CSV, it would automatically be created. I was wowed by it!!

    And now with layered navigation using the “anchor” tool for categories, we can easily setup “shop by options” and so I want the spreadsheet to have these “shop_by_price” | “shop_by_color” | “shop_by_style” | Etc…

    I’m told by some of our subcontracted developers that this would take 4 days at 8 hours per day to complete. So I gave them the goahead…. If someone here thinks they can do it in that time or want to see the import function when it’s done let me know.

    I’ll have it posted in the community… however you do that (sorry but Magento didn’t build their site as user-friendly as I thought it would be)

  27. Zep says:

    Count me in on this bad boy, Magento’s lacking product import functionality is sometimes fustrating.

  28. Josh Pratt says:


    Help could certainly be helpful. What type of help might you be able to offer? What’s your skill set? Most help is probably needed from a PHP pro (OOP).

  29. Scott Swabey says:

    Hey Josh, looks like you’re filling a very urgently required need here. Would love to help out if needed.

  30. Georg Ringer says:

    as written in IRC I am very interested in your work, especially if you support upsell, crosssell,…

    of course donations are available

  31. Josh Pratt says:


    I’ve actually done a fair amount of work with syncing Magento with an application called CounterPoint. Let me know if there’s any questions I can help answer or anything.

  32. Cycore says:

    I am very interested in your script. I have to develop a solution to synchronize product data between Magento and a warehouse management app. Your script would be a great head start for me.

  33. Brian says:

    I am definitely interested. I have over 10,000 products. It took forever and a day to upload them. When I uploaded them, they weren’t finished. I uploaded them to get them on the site, but they didn’t have attributes for layered navigation.

    I’m going through the tedious process now of assigning products different attributes. However, I have to reimport all my products. Which takes forever. The Magento CSV import is border line nonfunctional. Please let me know when you have a script that can import custom attributes.

  34. Josh Pratt says:


    Great ideas! I meant to write about updating products on the roadmap. I do intend on writing in something like, if sku is found, update data instead of insert.

    I like your other ideas too. Not sure about managing categories – since this is specifically a product import script, but, who knows. Maybe along the way, it will make sense to add some stuff in. Thanks for the thoughts – much appreciated!

  35. Josh Pratt says:


    I can’t say from experience about Magento’s ability to handle 100k products. None of my clients have ever been that large. The most for us us around 7k at this point.

    Have you tried/considered the Magento extension that you can download to import your data from osCommerce? I tried it with a website that had a few thousand products, and it worked great. Brought in categories, products, customers, and orders. I didn’t have to touch a single bit of code to do that. (

    As for the script – I absolutely intent on releasing it – free. I might be able to give you some pointers or the code that I’ve written so far if you’d like an example (at least until my script is fully built). If so, just find me on the #magento IRC channel. Nickname is: Prattski.

  36. Vinai says:

    Great project – I’m very interested to hear updates!
    My projection is there will be very many websites using this tool…
    Here are a couple of random thoughts/ideas on the topic, probably not new to you but who knows…
    – Ability to Import and/or Update product catalog
    – Ability to manage categories (build tree from csv columns)
    – Force strict version dependency, since the table structure might be different between magento versions
    – Ability to testrun imports (display statistics: this import would have updated nnn products, inserted nnn products, …)
    – Logging
    – Ability to rollback import/updates

    Great project, thumbs up!

  37. Mike says:

    Do you plan to release your import script now or in the future?

    We’re thinking of switching from osCommerce (which we’ve modified a lot) to Magento, but we’re not sure if it can handle our catalog. I’ve been able to find folks on the Forums and IRC channel with “large” catalogs consisting of 15-20k products, but our catalog is roughly 60,000 with 2,500 categories, and we plan on adding another 40,000 products within 12 months. Our plan right now is to build a very rough import script just to see what happens, and if it works well, write something nicer.

  38. Nick says:

    Good stuff!

    I’ve used Easy Populate for osCommerce a lot which has a nice feature of using direct category names (for as many category levels as desired) in the CSV file. In this way you don’t have to bother with abstract category IDs.