Fatal Error: Allowed Memory Size of 134217728 Bytes Exhausted (CodeIgniter + XML-RPC)

Question

I have a bunch of client point of sale (POS) systems that periodically send new sales data to one centralized database, which stores the data into one big database for report generation.

The client POS is based on PHPPOS, and I have implemented a module that uses the standard XML-RPC library to send sales data to the service. The server system is built on CodeIgniter, and uses the XML-RPC and XML-RPCS libraries for the webservice component. Whenever I send a lot of sales data (as little as 50 rows from the sales table, and individual rows from sales_items pertaining to each item within the sale) I get the following error:

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 54 bytes)

128M is the default value in php.ini, but I assume that is a huge number to break. In fact, I have even tried setting this value to 1024M, and all it does is take a longer time to error out.

As for steps I've taken, I've tried disabling all processing on the server-side, and have rigged it to return a canned response regardless of the input. However, I believe the problem lies in the actual sending of the data. I've even tried disabling the maximum script execution time for PHP, and it still errors out.

Solution 1

Changing the memory_limit by ini_set('memory_limit', '-1'); is not a proper solution. Please don't do that.

Your PHP code may have a memory leak somewhere and you are telling the server to just use all the memory that it wants. You wouldn't have fixed the problem at all. If you monitor your server, you will see that it is now probably using up most of the RAM and even swapping to disk.

You should probably try to track down the offending code in your code and fix it.

Solution 2

ini_set('memory_limit', '-1'); overrides the default PHP memory limit.

Solution 3

The correct way is to edit your php.ini file. Edit memory_limit to your desire value.

As from your question, 128M (which is the default limit) has been exceeded, so there is something seriously wrong with your code as it should not take that much.

If you know why it takes that much and you want to allow it set memory_limit = 512M or higher and you should be good.

Solution 4

The memory allocation for PHP can be adjusted permanently, or temporarily.

Permanently

You can permanently change the PHP memory allocation two ways.

If you have access to your php.ini file, you can edit the value for memory_limit to your desire value.

If you do not have access to your php.ini file (and your webhost allows it), you can override the memory allocation through your .htaccess file. Add php_value memory_limit 128M (or whatever your desired allocation is).

Temporary

You can adjust the memory allocation on the fly from within a PHP file. You simply have the code ini_set('memory_limit', '128M'); (or whatever your desired allocation is). You can remove the memory limit (although machine or instance limits may still apply) by setting the value to "-1".

Solution 5

It's very easy to get memory leaks in a PHP script - especially if you use abstraction, such as an ORM. Try using Xdebug to profile your script and find out where all that memory went.

View more solutions

Comments

[ArcticZero]

I'll go try Xdebug. I have never used it before, so I'll have to read up on it. Thank you for replying! Hope I find the answer to this soon...

[troelskn]

Remember that PHP uses reference counting for managing memory. So if you have circular references, or global variables, those objects won't get recycled. That's usually the root of memory leaks in PHP.

[ArcticZero]

Xdebug shows that CI's Xmlrpc.php library is responsible for my memory leak. By any chance, would there be any issues with CodeIgniter's XML-RPC libraries that I should know about? I have tried disabling all processing server-side, and it still runs out of memory if I feed it enough data.

[troelskn]

I don't know/use CI, so I don't know. But you should probably try to find an object that isn't freed up after use - most likely because of a cyclic reference. It's detective-work.

[Alisso]

where should one change that?! I only find that line in php.ini

[FLY]

Not in Drupal7 there is no such string of code in settings.php

[DrCord]

In certain situations where you absolutely need something to complete and then change this back to a reasonable setting, this really helps.

[Alix Axel]

@williamcarswell; -1 is a value PHP understands as unlimited in this context.

[baldrs]

combined with max_execution_time = -1 this can consume all resources the server can spare.

[Ken Williams]

@ArseniuszŁozicki - it will also consume resources the server can't spare.

[Markus Schober]

Honestly, if your caching some serious amounts of data, this is the correct answer. 128M is not enough for certain scripts. 512M or 1024M will often be enough, but you have to decide case by case.

[Markus Schober]

Shame that this gets so many upvotes. Setting it to an accurate value, with either php.ini edits or ini_set, is a perfectly valid solution when people need more memory. Setting it to unlimited is a dangerous hack :(

[Yoseph]

@Jeff you are probably right 95% of the time. However, there are times when you actually do need more memory. For example, let's say your app is loading a massive amount of data into memory for processing (say a Bill of Material with 15k components). It is not always the case that the code is buggy, sometimes you just need a little bit more memory (e.g. 256M instead of 128M). However I agree that setting it to -1 is horribly bad. But adjusting the memory limit for reasonable situations at run-time is perfectly acceptable imho.

[user1767586]

@JeffDavis It's the best answer. I'm setting this a lot because I have PHP scripts I run on my desktop where I couldn't care less about the "dangerous" consequences. I just want to prevent the script from throwing that error, that's all.

[Markus Schober]

@user1767586 then set it to a sane value. You could prevent the script from throwing the error by setting it to 1024M. If this answer said ini_set('memory_limit', '1024M'); You could copy-paste that and be ok. By setting it to -1 you are setting yourself up to have a script that consumes all memory. Especially if you do this routinely. Putting "dangerous" in quotes doesn't make it any less dangerous. You really could hose your host server. Maybe start destroying data. I don't know, maybe lose your job? Sounds pretty dangerous to me. :|

[Alberto Gaona]

I use ORM (doctrine) all the time and I haven't managed to have a single memory leak.

[Basav]

Yeha, however try to avoid huge memory use, if the number of users are going to be more

[Admin]

memory_limit = -1 ; set in php.ini

[Lukas Lukac]

@pyrite yes you are right that sometimes a process requires more memory but you should increase the memory limit to some logical amount like 256MB as you said or 512MB why not BUT not -1 ;)

[mathielo]

memory_get_usage(true) may come handy if you are trying to figure out precisely where the memory outage is happening. This way you can try to improve your script performance without excessive memory consumption.

[Esolitos]

@jeff I fully agree, a value of -1 could be useful only in dev environments to test purposes.

[albanx]

this is not the correct way even this is much upvoted. this will lead to poor performance code.

[Chris Baker]

This is the only answer here that advises actually addressing the problem. The other answers crank up memory to bandage over a symptom and ignore the disease.

[jfmercer]

With all due respect, this answer should be down-voted into oblivion. This solution "fixes" one problem, but introduces a host of other problems. It's like trading away the one devil you've got in return for dozens of more devils. As has already been suggested, (1) increase global PHP memory availability in memory_limit in php.ini and/or (2) diagnose the memory leak.

[floriank]

@Pyrite in the cases you named for the remaining 5% read the data in chunks and use a worker to process it instead of using more memory. This solution will scale as well while your suggestion won't work except your keep stuffing more and more memory into your server over the time if the data grows.

[akarthik10]

Sad to see that the answer for +161 votes and -3 votes is the same :(

[Flimm]

@YumYumYum That removes the memory_limit, which you only want if you're monitoring memory usage some other way. The OS will kill the process if it's taking a huge amount of memory at some point any way.

[user3806549]

This is a very bad idea

[Gogol]

you mean ini_set('memory_limit', '8192M'); ?

[HostMyBus]

Thank you I did not think to check if someone had set the value in .htaccess which was overriding php.ini and I couldnt figure out why +1

[Steve C]

this worked for me. love one line solutions. +1 for simplicity

[Stepchik]

In most common way this trouble in ORM when you try to fetchAll data that much more php memory limit. For example when you try to generate monthly report.

[Stepchik]

There better solution not to store all data in memory and use database cursors and php yield, fetch data row by row and place it in your report. Here is small example with cursors and yield generators github.com/stepchik/stuff/tree/master/php_memory_problem

[kiltek]

Always think before you use this, if it is really neccessary.

[Yogesh Trivedi]

what to track on the fresh installation, error log just says that "Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 4096 bytes)"

[Farfromunique]

set_include_path(get_include_path() . get_include_path().'/phpseclib'); This will add the path '/phpseclib' once for each file that has the line... so it can add it many times! I'd suggest putting it in a settings file and include_once the settings file.

[Mantisse]

Works really well with high loaded production program, thanks infinitely !

[AllisonC]

There is also no string in settings.php for drupal 6

[toddmo]

And another tip: you can put die('here') in your code and move that statement around to see where the recursion starts.

[Matthew Poer]

What a luxury it would be to have time to go and optimize a script for something like that. Or research and compare and learn ETL tools or some such. In the real world, we jack the memory allowance way up, do the thing, and move on.

[halfer]

This appears to say the same thing as many existing answers. It is best to only add an answer to a popular question only if the new material offers something novel.

[Peter Mortensen]

PHP.ini? Isn't it php.ini?

[Ashiq]

In case anyone with laravel project wonder where to put it! If the code is executing in web page call, then you need to put it in index.php in the very first line after <?php starts. If the code is running from any artisan command like queue, then you need to add this in artisan file in same position as told before.....

[chromechris]

So if you are running a script that uses a lot of memory, but you only need to run it once, can you just increase the memory limit for the process at the time of execution, then lower your memory limit again after your one-time script runs?

[mickmackusa]

I had a similar occurrence on Google Chrome today. I was extremely skeptical of this answer ...however, it did reveal that my byte exhaustion disappeared after I opened an incognito window and fired the same script again! The researching continues.

[Lamellama]

I was recieving this error locally within vscode, caused by my php extensions so it worked perfectly.

[ehsan asarzadeh]

how to track it in localhost?

[The Anh Nguyen]

For anyone who needs set temporary with commands: php -d memory_limit=256M your_php_file.php or php -d memory_limit=256M artisan ...

[Ariful Islam]

Perfect answer . I like that. Cutting head is not a solution. it might be dangerous

[zdanman]

This only happens when I run script as a cron job. If I run it directly... no issue. Please any advice?

[ChronoLogic]

For those looking for answers... I'd skip this one. I see no way that the memory consumption of a web browser (such as chrome or firefox) on the client machine has anything at all to do with the memory consumption used by a php script on the server machine. If anyone has noticed a server memory issue disappear when something changed on the client it's without question a coincidence. That's not to say that some client request for a lot of stuff could cause a server (with bad code) to run out of memory, or for the client (with different bad code) to also run out of memory.

[Yogi Arif Widodo]

php -r "echo ini_get('memory_limit').PHP_EOL;" 2048M i still get the an error Allowed memory size of 134217728 bytes exhausted (tried to allocate 16384 bytes) but the information about tried to allocate is from xx16384 into 16388 did u know why ?