Composer memory exhausted on shared host

Question

A previously working sandbox on a shared FreeBSD host now fails to run composer update or install with a memory exhausted error. Before yesterday I was able to run php ./composer.phar update without a problem. I do not believe that more memory is required. I can update the project on a Windows system without any issue. What variables other than memory size contribute to the memory exhausted error?

To test this I moved the contents of .../vendor and the composer.lock file to a different directory then ran php ./composer.phar install --prefer-dist. [Composer is in the project directory so that I can update it locally rather than expect the host master to keep it updated.] The above command results in:

% php ./composer.phar install --prefer-dist
Loading composer repositories with package information
Updating dependencies (including require-dev)
PHP Fatal error:  Allowed memory size of 1073741824 bytes exhausted (tried to allocate 134217728 bytes) in phar:///home/projectmana/www3.projectmana.org/composer.phar/src/Composer/DependencyResolver/Solver.php on line 220

Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 134217728 bytes) in phar:///home/projectmana/www3.projectmana.org/composer.phar/src/Composer/DependencyResolver/Solver.php on line 220

Composer is version 1.20

composer.json used:

{
    "name": "truckee/projectmana",
    "license": "MIT",
    "type": "project",
    "description": "Project MANA administrative application",
    "autoload": {
        "psr-0": {
            "": "src/"
        }
    },
    "repositories": [
        {
            "type": "package",
            "package": {
                "name": "jquery/jquery",
                "version": "1.11.1",
                "dist": {
                    "url": "https://code.jquery.com/jquery-1.11.1.js",
                    "type": "file"
                }
            }
        }
    ],
    "require": {
        "braincrafted/bootstrap-bundle": "~2.0",
        "doctrine/doctrine-bundle": "~1.4",
        "doctrine/doctrine-fixtures-bundle": "^2.3",
        "doctrine/orm": "^2.4.8",
        "friendsofsymfony/user-bundle": "~2.0@dev",
        "incenteev/composer-parameter-handler": "~2.0",
        "javiereguiluz/easyadmin-bundle": "~1.1",
        "jms/security-extra-bundle": "~1.5",
        "jquery/jquery": "1.11.*",
        "knplabs/knp-menu-bundle": "~2.0",
        "nelmio/alice": "^2.1",
        "oyejorge/less.php": "~1.5",
        "paragonie/random_compat": "^2.0",
        "php": ">=5.3.9",
        "psliwa/pdf-bundle": "dev-master",
        "sensio/distribution-bundle": "~4.0",
        "sensio/framework-extra-bundle": "^3.0.2",
        "symfony/assetic-bundle": "dev-master",
        "symfony/monolog-bundle": "~2.4",
        "symfony/swiftmailer-bundle": "~2.3",
        "symfony/symfony": "2.8.*",
        "twbs/bootstrap": "3.0.*",
        "twig/extensions": "1.0.*"
    },
    "scripts": {
        "post-install-cmd": [
            "Incenteev\\ParameterHandler\\ScriptHandler::buildParameters",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::buildBootstrap",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::clearCache",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installAssets",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installRequirementsFile",
            "Braincrafted\\Bundle\\BootstrapBundle\\Composer\\ScriptHandler::install"
        ],
        "post-update-cmd": [
            "Incenteev\\ParameterHandler\\ScriptHandler::buildParameters",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::buildBootstrap",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::clearCache",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installAssets",
            "Sensio\\Bundle\\DistributionBundle\\Composer\\ScriptHandler::installRequirementsFile",
            "Braincrafted\\Bundle\\BootstrapBundle\\Composer\\ScriptHandler::install"
        ]
    },
    "config": {
        "bin-dir": "bin"
    },
    "minimum-stability": "stable",
    "extra": {
        "symfony-app-dir": "app",
        "symfony-web-dir": "htdocs",
        "incenteev-parameters": {
            "file": "app/config/parameters.yml"
        },
        "branch-alias": {
            "dev-master": "2.3-dev"
        },
        "repositories": [
            {
                "type": "composer",
                "url": "http://packages.zendframework.com/"
            },
            {
                "type": "composer",
                "url": "http://packagist.org/"
            }
        ]
    },
    "require-dev": {
        "liip/functional-test-bundle": "^1.4",
        "symfony/phpunit-bridge": "^3.0"
    }
}

Edit: Available memory:

% php -r "echo(ini_get('memory_limit'));"
128M

Requested free -m: % free -m free: Command not found.

with -vv --profile:

% composer update -vv --profile
[9.9MB/0.01s] Loading composer repositories with package information
[10.5MB/0.80s] Updating dependencies (including require-dev)
PHP Fatal error: ...

Edit #2, fatal error (using composer 1.1.0) continued, as it differs from that above:

Allowed memory size of 1073741824 bytes exhausted (tried to allocate 32 bytes) in phar:///usr/local/bin/composer.phar/src/Composer/DependencyResolver/RuleWatchGraph.php on line 52

Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 32 bytes) in phar:///usr/local/bin/composer.phar/src/Composer/DependencyResolver/RuleWatchGraph.php on line 52

Solution 1

Although I thought I'd tried this before, I achieved success with php -d memory_limit=-1 ./composer.phar update.

FWIW, here's the output of % php -d memory_limit=-1 ./composer.phar -vv --profile update:

[9.7MB/0.01s] Loading composer repositories with package information
[10.3MB/0.72s] Updating dependencies (including require-dev)
[1094.4MB/57.65s] Dependency resolution completed in 8.075 seconds
[1094.4MB/57.72s] Analyzed 8525 packages to resolve dependencies
[1094.4MB/57.72s] Analyzed 514764 rules to resolve dependencies
[1095.7MB/57.76s] Dependency resolution completed in 0.002 seconds
[287.5MB/66.04s] Generating autoload files
[288.1MB/67.86s] > post-update-cmd: Incenteev\ParameterHandler\ScriptHandler::buildParameters
[288.3MB/71.19s] Updating the "app/config/parameters.yml" file
[288.6MB/71.92s] > post-update-cmd: Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::buildBootstrap
[288.7MB/86.31s] > post-update-cmd: Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::clearCache
[288.7MB/101.92s]
[288.7MB/101.92s]  // Clearing the cache for the dev environment with debug true
[288.7MB/101.92s]
[288.7MB/102.05s]                                                               
 [OK] Cache for the "dev" environment (debug=true) was successfully cleared.    
[288.7MB/102.05s]                                                               

[288.7MB/102.12s] > post-update-cmd: Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::installAssets
[288.7MB/104.14s]
[288.7MB/104.14s]  Installing assets as hard copies.

[288.7MB/104.31s]  --- -------------------------- ----------------
[288.7MB/104.31s]       Bundle                     Method / Error
[288.7MB/104.31s]  --- -------------------------- ----------------
[288.7MB/104.31s]   ✔   FrameworkBundle            copy
[288.7MB/104.31s]   ✔   EasyAdminBundle            copy
[288.7MB/104.31s]   ✔   SensioDistributionBundle   copy
 --- -------------------------- ----------------
[288.7MB/104.31s]
[288.7MB/104.31s]  ! [NOTE] Some assets were installed via copy. If you make changes to these assets you have to run this command again.

[288.7MB/104.31s]                                                               
 [OK] All assets were successfully installed.                                   

[288.7MB/104.31s]
[288.7MB/104.38s] > post-update-cmd: Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::installRequirementsFile
[288.9MB/104.43s] Deprecation Notice: The callback Braincrafted\Bundle\BootstrapBundle\Composer\ScriptHandler::install declared at /home/projectmana/www3.projectmana.org/vendor/braincrafted/bootstrap-bundle/Braincrafted/Bundle/BootstrapBundle/Composer/ScriptHandler.php accepts a Composer\Script\CommandEvent but post-update-cmd events use a Composer\Script\Event instance. Please adjust your type hint accordingly, see https://getcomposer.org/doc/articles/scripts.md#event-classes in phar:///home/projectmana/www3.projectmana.org/composer.phar/src/Composer/EventDispatcher/EventDispatcher.php:289
[288.9MB/104.43s] Stack trace:
[288.9MB/104.43s]  phar:///home/projectmana/www3.projectmana.org/composer.phar/src/Composer/EventDispatcher/EventDispatcher.php:247
[288.9MB/104.43s]  phar:///home/projectmana/www3.projectmana.org/composer.phar/src/Composer/EventDispatcher/EventDispatcher.php:209
[288.9MB/104.43s]  phar:///home/projectmana/www3.projectmana.org/composer.phar/src/Composer/EventDispatcher/EventDispatcher.php:95
[288.9MB/104.43s]  phar:///home/projectmana/www3.projectmana.org/composer.phar/src/Composer/Installer.php:297
[288.9MB/104.43s]  phar:///home/projectmana/www3.projectmana.org/composer.phar/src/Composer/Command/UpdateCommand.php:174
[288.9MB/104.43s]  phar:///home/projectmana/www3.projectmana.org/composer.phar/vendor/symfony/console/Command/Command.php:259
[288.9MB/104.43s]  phar:///home/projectmana/www3.projectmana.org/composer.phar/vendor/symfony/console/Application.php:844
[288.9MB/104.43s]  phar:///home/projectmana/www3.projectmana.org/composer.phar/vendor/symfony/console/Application.php:192
[288.9MB/104.43s]  phar:///home/projectmana/www3.projectmana.org/composer.phar/src/Composer/Console/Application.php:231
[288.9MB/104.43s]  phar:///home/projectmana/www3.projectmana.org/composer.phar/vendor/symfony/console/Application.php:123
[288.9MB/104.43s]  phar:///home/projectmana/www3.projectmana.org/composer.phar/src/Composer/Console/Application.php:104
[288.9MB/104.43s]  phar:///home/projectmana/www3.projectmana.org/composer.phar/bin/composer:43
[288.9MB/104.43s]  /home/projectmana/www3.projectmana.org/composer.phar:24
[288.9MB/104.44s] > post-update-cmd: Braincrafted\Bundle\BootstrapBundle\Composer\ScriptHandler::install
Copied icon fonts to /home/projectmana/www3.projectmana.org/app/../web/fonts.
[288.9MB/105.09s] Memory usage: 288.87MB (peak: 1095.93MB), time: 105.09s

Solution 2

you can try this, I'm success

php -d memory_limit=-1 /usr/local/bin/composer/composer.phar require guzzlehttp/guzzle

this is my first answer!!!wow

Solution 3

Composer update use a lot memory to resolve dependencies

Your only option on server with so little memory is to provide composer.lock either by committing it to the repository or copy to server from continuous integration server.

There is no other way to run composer on server with 128mb ram

Comments

[geoB]

Thanks for your suggestion. What eventually worked was php -d memory_limit=-1 ./composer.phar update. As a test of your suggestion I ran php ./composer.phar update immediately after the successful update. The memory exhausted error occurred again. The magic appears to be in memory_limit=-1.

[geoB]

@A.L You may find the memory usage in my answer useful.

[Francesco Abeni]

memory_limit=-1 basically removes all memory limitations. So it works because you are using all memory available on the server, which is in some way like setting a very high limit. If your question was "why it is using so much memory" this is not an actual answer.

[geoB]

@FrancescoAbeni My question was whether there were other variables that affected the use of memory (implying something other than composer.json). The success of memory_limit=-1, along with the memory usage profile, strongly suggests that there are not other variables.

[Konrad Podgórski]

but I think that host provider won't like that you set memory limit to unlimited :)

[geoB]

I emailed the host with the effect of defeating the php memory limit. His reply: "I will trust you to be careful".

[Varsha Dhadge]

@geoB you should post that as answer. It worked for me thanks.

[Damodar Bashyal]

On Centos 7, your trick worked for me php -d memory_limit=-1 /usr/bin/composer require guzzlehttp/guzzle