PHP - Flushing While Loop Data with Ajax

php ajax loops while-loop flush

26,114

Solution 1

Using:

should do all you need in one php thread

EDIT

Take a look at nickb's answer, if you're looking for a way how to do this simply it would be following algorithm:

javascript opens process.php via ajax (which will do all the work AND print status reports), you have to look up whether jQuery ajax supports continuous loading
if user decides to stop refreshes you'll kill loading as show in provided link

In process.php:

ignore_user_abort(); // Script will finish in background
while(...){
  echo "Page: $i\n";
  ob_flush();
}

EDIT 2 requested example (bit of different and ugly, but simple). test_process.php:

// This script will write numbers from 1 to 100 into file (whatever happens)
// And sends continuously info to user
$fp = fopen( '/tmp/output.txt', 'w') or die('Failed to open');
set_time_limit( 120);
ignore_user_abort(true);

for( $i = 0; $i < 100; $i++){
    echo "<script type=\"text/javascript\">parent.document.getElementById( 'foo').innerHTML += 'Line $i<br />';</script>";
    echo str_repeat( ' ', 2048);
    flush();
    ob_flush();
    sleep(1);
    fwrite( $fp, "$i\n");
}

fclose( $fp);

And main html page:

<iframe id="loadarea"></iframe><br />
<script>
function helper() {
    document.getElementById('loadarea').src = 'test_process.php';
}
function kill() {
    document.getElementById('loadarea').src = '';
}
</script>

<input type="button" onclick="helper()" value="Start">
<input type="button" onclick="kill()" value="Stop">
<div id="foo"></div>

After hitting start lines as:

Line 1
Line 2

Appeared in the div #foo. When I hit Stop, they stopped appearing but script finished in background and written all 100 numbers into file.

If you hit Start again script starts to execute from the begging (rewrite file) so would parallel request do.

For more info on http streaming see this link

Solution 2

You're confused as to how PHP and AJAX interact.

When you request the PHP page via AJAX, you force the PHP script to begin execution. Although you might be using flush() to clear any internal PHP buffers, the AJAX call won't terminate (i.e., the response handlers won't be called) until the connection is closed, which occurs when the entire file has been read.

To accomplish what you're looking for, I believe you'd need a parallel process flow like this:

The first AJAX post sends a request to begin reading the file. This script generates some unqiue ID, sends that back to the browser, spawns a thread that actually does the file reading, then terminates.
All subsequent AJAX requests go to a different PHP script that checks the status of the file reading. This new PHP script sends the current status of the file reading, based on the unique ID generated in #1, then exits.

You could accomplish this inter-process communication through $_SESSION variables, or by storing data into a database. Either way, you need a parallel implementation instead of your current sequential one, otherwise you will continue to get the entire status at once.

Solution 3

Simpler solution should be using the native (vanila js) XHR object.

There is very sophisticated solutions out there, about long polling

The PHP:

<?php
header('Content-Type: text/html; charset=UTF-8');
if (ob_get_level() == 0) ob_start();
for ($i = 0; $i<10; $i++){
  echo "<br> Line to show.";
  echo str_pad('',4096)."\n";
  ob_flush();
  flush();
  sleep(2);
}
echo "Done.";
ob_end_flush();

The JS:

var xhr = new XMLHttpRequest();
xhr.open('GET', '/api/some_service.php', true);

xhr.send(null);
xhr.onreadystatechange = function() {
  if (xhr.status == 200) {
    if (xhr.readyState == XMLHttpRequest.LOADING){
      console.log('response',xhr.response);
      // this can be also binary or your own content type 
      // (Blob and other stuff)
    }
    if (xhr.readyState == XMLHttpRequest.DONE){
      console.log('response',xhr.response);
    }
  }
}

26,114

Author by

ThreaT

Updated on July 09, 2022

Comments

ThreaT almost 2 years

Using PHP, I would like to make a while loop that reads a large file and sends the current line number when requested. Using Ajax, I'd like to get the current line count and print it out onto a page. Using html buttons, I'd like to be able to click and activate or terminate a javascript thread that runs only ONCE and calls the ajax method.

I have given it a shot but for some reason, nothing prints unless I comment out the echo str_repeat(' ',1024*64); function and when it's commented out, it shows the entire loop result:

1 row(s) processed.2 row(s) processed.3 row(s) processed.4 row(s) processed.5 row(s) processed.6 row(s) processed.7 row(s) processed.8 row(s) processed.9 row(s) processed.10 row(s) processed.

In a single line instead of showing them in separate lines like:

1 row(s) processed.
2 row(s) processed.
3 row(s) processed.
4 row(s) processed.
5 row(s) processed.
6 row(s) processed.
7 row(s) processed.
8 row(s) processed.
9 row(s) processed.
10 row(s) processed.

Also I'm not sure how to terminate the JavaScript thread. So 2 problems in total:

 1. It's returning the entire While loop object at once instead of each time it loops.
 2. I'm not sure how to terminate the JQuery thread.

Any ideas? Below is my code so far.

msgserv.php

<?php

//Initiate Line Count
$lineCount = 0;

// Set current filename
$file = "test.txt";

// Open the file for reading
$handle = fopen($file, "r");

//Change Execution Time to 8 Hours
ini_set('max_execution_time', 28800);

// Loop through the file until you reach the last line
while (!feof($handle)) {

    // Read a line
    $line = fgets($handle);

    // Increment the counter
    $lineCount++;

    // Javascript for updating the progress bar and information
    echo $lineCount . " row(s) processed.";

    // This is for the buffer achieve the minimum size in order to flush data
    //echo str_repeat(' ',1024*64);

    // Send output to browser immediately
    flush();

    // Sleep one second so we can see the delay
    //usleep(100);
}

// Release the file for access
fclose($handle);

?>

asd.html

<html>
    <head>
        <script src="http://code.jquery.com/jquery-latest.min.js" type="text/javascript" charset="utf-8"></script>

        <style type="text/css" media="screen">
            .msg{ background:#aaa;padding:.2em; border-bottom:1px #000 solid}
            .new{ background-color:#3B9957;}
            .error{ background-color:#992E36;}
        </style>

    </head>
    <body>

    <center>
        <fieldset>
            <legend>Count lines in a file</legend>
            <input type="button" value="Start Counting" id="startCounting" />
            <input type="button" value="Stop Counting!" onclick="clearInterval(not-Sure-How-To-Reference-Jquery-Thread);" />
        </fieldset>
    </center>

    <div id="messages">
        <div class="msg old"></div>
    </div>

    <script type="text/javascript" charset="utf-8">
        function addmsg(type, msg){
            /* Simple helper to add a div.
        type is the name of a CSS class (old/new/error).
        msg is the contents of the div */
            $("#messages").append(
            "<div class='msg "+ type +"'>"+ msg +"</div>"
        );
        }

        function waitForMsg(){
            /* This requests the url "msgsrv.php"
        When it complete (or errors)*/
            $.ajax({
                type: "GET",
                url: "msgsrv.php",
                async: true, /* If set to non-async, browser shows page as "Loading.."*/
                cache: false,
                timeout:2880000, /* Timeout in ms set to 8 hours */

                success: function(data){ /* called when request to barge.php completes */
                    addmsg("new", data); /* Add response to a .msg div (with the "new" class)*/
                    setTimeout(
                    'waitForMsg()', /* Request next message */
                    1000 /* ..after 1 seconds */
                );
                },
                error: function(XMLHttpRequest, textStatus, errorThrown){
                    addmsg("error", textStatus + " (" + errorThrown + ")");
                    setTimeout(
                    'waitForMsg()', /* Try again after.. */
                    "15000"); /* milliseconds (15seconds) */
                },
            });
        };

        $('#startCounting').click(function() {
            waitForMsg();
        });
    </script>

</body>
</html>

test.txt

ThreaT about 12 years

Hi - Thanks a lot. I tried your suggestions but none of them work for me. Could you possibly save the 3 files I provided and tell me if they work for you? If so, could you share what you found?
ThreaT about 12 years

Hi - Thanks a lot. I tried your suggestions but none of them work for me. Could you possibly save the 3 files I provided and tell me if they work for you? If so, could you share what you found?
ThreaT about 12 years

Hey, thanks again - I tried your new suggestions and still no luck yet... Did they work for you?
Vyktor about 12 years

@user1191027 I've never tried :P those are things that should work (based on my experiences on other things) and it's your (and others here) job to point out possible bugs and help me reach perfect working answer. I tried to provide you with an idea how to do this.
ThreaT about 12 years

Well in theory your suggestions seem flawless to me, but I'm new to PHP and I'm having a difficult time putting them together so if you could possibly download my examples and show me where I went wrong, I'd greatly appreciate that.
Vyktor about 12 years

@user1191027 here you go (you're lucky my karma is so bad ;D)
ThreaT about 12 years

Awesome, works really well - thank you so much! For some reason I get the error "Notice: ob_flush() [ref.outcontrol]: failed to flush buffer. No buffer to flush" - I googled around and apparently it's because you have to first ob_start() - but when I use that then it doesn't print out line by line. Any idea why this type of behaviour would occur? Does it do that with you too?
Vael Victus over 8 years

An example of why you should post the content inside an answer rather than simply linking.
Didier Sampaolo over 8 years

Fully understood. I'll do that instead from now on.
Samuel Gfeller over 2 years

This is the way to go. This answer is underrated.
Ed_Johnsen over 2 years

This simple solution worked for me. One tiny detail to keep in mind is that the xhr.response variable will contain redundant info at every step after the first response in the sequence. Basically, you'd receive "A" then "AB" then "ABC" if the php was flushing "ABC" in steps.
EasyWay about 2 years

@Ed_Johnsen Do you have any idea how to get unique response each not just concatinate?
JohnnyJS about 2 years

XMLHttpRequest object does not seems to have a property to get you the bytes just received. You can develop your own function, that will eliminate the known bytes from the start, to get you only the new ones.
Giovanni Palerma about 2 years

@EasyWay store the previous loop response in a variable and string replace it on the next loop.