ssossossosso

Forums

Covering OroCRM topics, including community updates and company announcements.  Subscribe

This topic contains 7 replies, has 4 voices, and was last updated by  Rogyar 1 year, 10 months ago.

  • Creator
    Topic
  • #13950

    Rogyar
    Participant

    Hello Guys.
    We had attempts to integrate Magento stores with small amount of users/orders in the past with no issues. Now we are attempting to import a quite huge amount of Magento entities (700k orders, 500k customers). As result, we had throughput near ~15-20 entities per minute on the initial sync (Magento->Orocrm).
    First of all, we suspected the bottleneck is on the Magento side. After some investigations we got that each SOAP request takes ~1 sec, so, the problem is not here. Then we have checked the OroCRM server state and got 100% CPU load by PHP process. After this a new OroCRM instance has been installed on a more powerful server (c3.2xlarge AWS instance with 8 CPUs). As result, we have ~20-25 entities per minute and 100% CPU load by PHP process. As we can see, the bottleneck somewhere on the OroCRM side. Since we have tried different server configurations, must assume that the hardware/system configuration problem is not supposed to be here.

    Current states are the following:
    – OroCRM (1.8.2) and Magento (1.9.1) installed on different servers.
    – OroCRM uses a separate RDS MYSQL instance.
    – Magento has OroCRM bridge extension (1.1.4.3) installed, WSDL cache enabled.
    – Xdebug is disabled on both servers.

    Maybe there were some changes in the recent OroCRM versions connected to the import and there are some workarounds to speed up the integration? Any help is very appreciated. Thanks.

Viewing 7 replies - 1 through 7 (of 7 total)
  • Author
    Replies
  • #13951
    Artem Liubeznyi
    Artem Liubeznyi
    Oro Core

    Hi Rogyar,

    We have released a new 1.2 version of OroCRM Bridge along with 1.7 OroCRM release. The current version of Bridge is 1.2.3.0 released just two days ago.

    Please try updating the Bridge to the newest version and check if the problem persists. Thank you.

    #13955

    Rogyar
    Participant

    Thanks for your reply. We have upgraded the OroCRM Bridge extension but the situation was not changed. I guess, there’s no problem with users/orders fetching from Magento. The users/orders processing in OroCRM causes 100% CPU loading and still quite slow. We need to wait about 2 months for the import to be completed on the powerful system. That fact does not sound like a solution 🙂

    Thanks.

    #13971

    dmytro
    Participant

    We have also struggled with somehow slow connection on Oro’s end.. And due to the fact that no other export methods are available, we had to stop the process.

    If anyone figures out a way to move 120000 orders in oro without waiting for 1 month – please hit me @ dmytro@kateandkimi.com .

    Thanks,
    D

    #13972
    Dima Soroka
    Dima Soroka
    Oro Core

    Hi guys

    We are investigating the root cause of this issue

    #13973

    dmytro
    Participant

    Thank you for looking into it.

    Please keep us posted.

    D

    #13974

    dmytro
    Participant

    if you’d like more details feel free to contact me on my email.

    #14065

    Rogyar
    Participant

    Thanks Guys.
    We have increased the import performance a little bit by disabling the search indexer in OroCRM. Also, we have noticed that if you periodically stops and runs the integration process (we are running the process via CLI) – the things go faster. In the same time, the process memory consumption is not high. Anyway, after 4 days of import we have 300k+ entities imported (20% of the total) and that fact is still sad..

Viewing 7 replies - 1 through 7 (of 7 total)

You must be logged in to reply to this topic.