Multi-Threading in Laravel

Question

I am running into a problem where my database calls are slowing up the page load significantly. I am populating multiple charts from elections data, and my table contains around 1 million rows, and I have to query this data multiple times in each methods inside the getCharts() method.

I'am using this to pass the return data to JavaScript.

These charts gets re-populated when you click on a data point. So if you click on a point i.e ('democrat) it will reload the page and call these methods again.

What I am asking is wether it is possible to do something like this in native PHP. The server is running PHP 5.2 on linode.

foreach(function in getChartsMethod){
     Start a child thread to process the function. 
}  
join the threads. 
reload the page. 


public function getCharts(){
        $this->get_cast_chart();
        $this->get_party_chart();
        $this->get_gender_chart();
        $this->get_age_chart();
        $this->get_race_chart();
        $this->get_ballot_chart();
        $this->get_county_chart();
        $this->get_precinct_chart();
        $this->get_congressional_district_chart();
        $this->get_senate_district_chart();
        $this->get_house_district_chart();
        return view('elections.index');
    }

Sample method

public function get_party_chart(){
        $query = DB::table($this->tableName)
            ->select('party', DB::raw('count(*) as numVotes'))
            ->groupBy('party')
            ->orderBy('numVotes', 'ASC');

        if ($this->filterParameters != null){
            $query = $query->where(key($this->filterParameters), $this->operator, current($this->filterParameters));
        }
        $chartData = $query->get();
        $chartData = $this->prepare_data_for_chart($chartData, 'party', 'numVotes');
        JavaScript::put(['partyChart' => $chartData]);

    }

Solution 1

Multithreading is possible in PHP which has been discussed on Stackoverflow before: How can one use multi threading in PHP applications

However I don't know how much multithreading will help you here. How much data are we talking, what kind of accuracy are you looking for in the charts? How many charts are being shown? A lot of the solution will come from context.

If it were me, I would have a background schedule pre-processing the "huge" amounts of data into something more useful/usable on the frontend. Again useful/usable entirely depend on the context. When there is an actual page request, you just need to pass down the processed data, versus actual "live" data.

Again, again, this will entirely depend on the context of your application. Perhaps you need by-the-minute data, maybe you don't.

Solution 2

Chris is right - it all depends on your application, but if you are are getting to the point where application is not usable, you should look in into cashing your results in redis or memcached.

Comments

[Yasin Yaqoobi]

The table sizes are roughly 1 million rows and all of these charts show count of votes. Roughly speaking the only difference is how I group them (showVotes by gender, party, age, etc..). This data is uploaded/updated every day for three months until the end of the election day and after that there are no changes. The problem with using scheduler is that I have to run a lot of tasks in the background because there are many variations of this data. i.e If you click on age 60, it will reload the page with election data only for 60 years old.

[Chris]

If you are filtering down the data to say "60 year olds" for example, the data set shouldn't be too large anymore, and you can still do server side normalisation to send through a much smaller packet of data to the frontend. Again its very hard to answer this question as its very broad and depends on context

[Yasin Yaqoobi]

thank you. What do you think about using scheduler and laravel.com/docs/5.1/cache to preemptively cache the data and flush everything when the data is uploaded ? I guess I am a bit confused on how to make the first page load fast.