How to avoid out of memory error in a browser due to too many ajax calls

Question

I'm building a web application that needs to make about 28000 database calls using the jquery ajax shortform all at once.

It gets through about 6000 of the calls fine, but then the browser gives me about 20000 of the following error (one for each call) in the browser console:

POST (my database call) net : : ERR_INSUFFICIENT_RESOURCES

Does anyone know how to get around this? Maybe to create a buffer or something?

Thanks!

edit 1 : adding some code:

Aright, so the user would fill in some values (say GHI > 4500, aspect between 157.5 and 202.5)

The following call would be made:

loadAllData('ghi', 4500, findIdealPoints);

This call leads to this function:

function loadAllData(type, above, callback){
var data = {};
$.post('php/getIdealData.php?action=get&type='+type+'&above='+above, data, callback);

}

which runs this query in PHP:

 "SELECT  `GHI` ,  `lat` ,  `long` 
    FROM solar
    WHERE  `GHI` >'{$_GET['above']}' ORDER BY `lat`,`long`;";

That returns about 28880 records in an array in JSON format and calls the callback method which does the following:

function findIdealPoints(data){
var i = 0;
while (i < data.length){
    loadAspectWithinRange('aspect', data[i]['lat'], data[i]['long'], 10, compareWithAspect);
    i++;
}     

}

Which runs this php query:

"SELECT `aspect`,
            `lat`, `long`, distance_in_km
            FROM (
                SELECT `aspect`, `lat`, `long`,r,
                (6378.10 * ACOS(COS(RADIANS(latpoint))
                * COS(RADIANS(`lat`))
                * COS(RADIANS(longpoint) - RADIANS(`long`))
                + SIN(RADIANS(latpoint))
                * SIN(RADIANS(`lat`)))) AS distance_in_km
            FROM aspect
            JOIN (
                SELECT  '{$_GET['lat']}'  AS latpoint, '{$_GET['long']}' AS longpoint, 10.0 AS r
            ) AS p
            WHERE `lat`
                BETWEEN latpoint  - (r / 111.045)
                AND latpoint  + (r / 111.045)
            AND `long`
                BETWEEN longpoint - (r / (111.045 * COS(RADIANS(latpoint))))
                AND longpoint + (r / (111.045 * COS(RADIANS(latpoint))))
            AND `aspect`
                BETWEEN '{$_GET['lowA']}' AND '{$_GET['highA']}'
            ) d
            WHERE distance_in_km <= r
            ORDER BY distance_in_km";

and goes to the callback function that runs this:

function compareWithAspect(data){
var idealPoints =[];
for (var i=0; i<data.length; i++){
            idealPoints.push(new google.maps.LatLng(data[i]['lat'], data[i]['long']));
        }

        if (idealPoints.length > 1){
            makePolygon(idealPoints)
        }
}

and makePolygon just draws on the map using the Google Maps API.

I know it's a lot and seems convoluted, I would love it if anyone could show me a better way to do this!

Thanks again

Solution

You could do something like this.

function findIdealPoints(data){
   var i = 0;
    while (i < data.length){
       loadAspectWithinRange('aspect', data[i]['lat'], data[i]['long'], 10,          
     compareWithAspect);
    i++;
}

Instead of doing an Ajax call for each occurrence send the data object to your call

 loadAspectWithinRange('aspect',data,10,compareWithAspect)

Then in the Ajax request send the array of objects to your service and retrieve the results for all of them instead of one by one.

$.ajax({
   url:"...",
   data:{
       attr1:'aspect',
       points: data(here is the array retrieved from "getIdealData.php")
       attr2: 10
    },
   success:function(data){
      compareWithAspect(data)
   }
})

In the server side processing build an array of the objects for all the element on the getIdealData.php points.

This will be better instead of doing an Ajax for each element

Comments

[Mo Tawakol]

Thank you very much! that should actually be much more efficient than what I was doing before. edit: I'll mark it correct when I try it :)

[Mo Tawakol]

Actually this doesn't work out for me because if I do it this way, it just draws a huge connected mess of lines that don't represent anything. Each point needs to reach the polygon drawer seperately. I think however, that I will take a different approach: I will make both calls and do the distance checking/logic in the javascript as opposed to making so many database calls.

[Eduardo Quintana]

If you need to draw a polygon for each of the 28800 calls why not to send them and return something like a List of List in this way you could add to the first list one List of LatLon objects representing the results for the first query send in that way you will have a List<List<LatLon>> and with this you will have the data ordered at least by each polygon required data in a list.

[Mo Tawakol]

I tried that but it's actually slower to run it on the server side when testing with smaller samples.. the 28800 is the max case scenario. Also, since it takes so much time, there's no way to indicate a completion percentage to a potential user. This would make it less usable than if it took 20 minutes to complete.. (I got it working on firefox, it only crashes in Chrome now)