How to use PHP to delete X number of lines from the beginning of a text file?

Question

Building my first ember app based on ember app kit.

I wanted to test the current route after transition - and found just what I need in ember docs - currentRouteName, currentURL and currentPath helper functions.

However, if I use these functions i get a "ReferencError: currentRouteName is not defined".

I stumbled upon this pull request that I suppose has the initial implementation of the same thing - but was closed in favor of adding this functionality to ember proper... For the time being, I copied the code from this PR and it does indeed get picked up and pass my tests.

Question: how can I utilize all the functions defined in my copy of ember.js in testing? These route helper functions are defined alongside with visit and click functions, which my tests seem to pick up without issue. Or, are my tests picking these functions up from somewhere else?

Solution 1

$x_amount_of_lines = 30;
$log = 'path/to/log.txt';
if (filesize($log) >= $max_size)) {
  $file = file($log);
  $line = $file[0];
  $file = array_splice($file, 2, $x_amount_of_lines);
  $file = array_splice($file, 0, 0, array($line, "\n")); // put the first line back in
  ...
}

edit: with correction from by rcar and saving the first line.

Solution 2

Use the SPL, Luke

PHP 5 comes with plenty of iterators goodness :

<?php

$line_to_strip = 5;
$new_file = new SplFileObject('test2.log', 'w');

foreach (new LimitIterator(new SplFileObject('test.log'), $line_to_strip) as $line)
    $new_file->fwrite($line);    

?>

It's cleaner that what you can do while messing with fopen, it does not hold the entire file in memory, only one line at a time, and you can plug it and reuse the pattern anywhere since it's full OO.

Solution 3

This is a text-book problem of log files, and I would like to propose another solution.

The problem with the "removing lines at the beginning of files" approach is that adding new lines becomes extremly slow, once it has to remove the first lines for every new lines it's writing.

Normal log file appending only involves writing a few more bytes at the end of the file in the file system (and once in a while it has to allocate a new sector, which results in extensive fragmentation - why log files usually are).

But the big problem here is when you are removing a line in the beginning for every row written. The entire file must first be read into memory and then rewritten resulting in huge ammount of I/O to the harddrive (in comparision). To make matters worse, the "split into PHP array and skip first rows" solutions here are extremly slow due to the nature of PHP arrays. This is not a problem if the log file size limit is very small or if it is written to unoften, but with a lot of writes (as in the case with log files), the same huge operation has to be done a lot of times resulting in major performance drawbacks.

This can be imagined as parking cars on a line with space for 50. Parking the first 50 cars is quick, just drive in behind the car infront and done. But when you come to 50, and the car at the front (beginning of file) must be removed you have to drive the 2'nd car to the 1'st position, 3rd to 2nd and so on, before you can drive in with the last car on the 50'th position. (And this must be repeated for every new car you want to park!)

My suggestion is instead saving to diffrent log files, datewise, and then store a maximum of 30 days back etc. Thus taking advantage of the filesystem, which has already solved this problem perfectly well.

Solution 4

You could use the file() function to read the file into an array of lines, then use array_slice() to remove the first X lines.

$X = 100; // Number of lines to remove

$lines = file('log.txt');
$first_line = $lines[0];
$lines = array_slice($lines, $X + 2);
$lines = array_merge(array($first_line, "\n"), $lines);

// Write to file
$file = fopen('log.txt', 'w');
fwrite($file, implode('', $lines));
fclose($file);

Solution 5

Here's a ready to go function

<?php
//--------------------------------
// FUNCTION TO TRUNCATE LOG FILES
//--------------------------------
function trim_log_to_length($path,$numHeaderRows,$numRowsToKeep){
    $file = file($path);
    $headerRows = array_slice($file,0,$numHeaderRows);
    // if this file is long enough were we should be truncating it
    if(count($file) - $numRowsToKeep > $numHeaderRows){
        // figure out the rows we wanna keep
        $dataRowsToKeep = array_slice($file,count($file)-$numRowsToKeep,$numRowsToKeep);
        // write the file
        $newFileRows = array_merge($headerRows,$dataRowsToKeep);
        file_put_contents($path, implode($newFileRows));
    }
}
?>

View more solutions

Comments

[Kingpin2k]

Which version of Ember are you using?

[apprenticeDev]

Hey @kingpin2k! Using EAK - in my bower.json I have "ember": "~1.3.0-beta.4", and when I look in vendor/ember/ember.js it is version 1.3.1.

[warnabas]

The problem with this is then, depending on how many lines is written, the dates/times will be out of order. Also, I foresee the file writing to the same lines over and over.

[warren]

Yes, that could be an issue (writing the same data repeatedly), depending on how large the file is - this is certainly a brute-force method :) .. why would date/times be out of order? unless during the read in php the file was appended-to, I don't think you would see that

[warnabas]

Fantastic idea, but for my particular needs, this is unacceptable. I have a small log file, that probably wont be written to much/often. But I'll keep this in mind for future projects.

[apprenticeDev]

Ahh, so exposing it globally is done with helper('currentRouteName', currentRouteName); and so forth. That makes sense. Thank you for hunting it down!

[apprenticeDev]

Also looks like they are conditionally included in 1.4 beta - although not to be seen (yet) in builds.emberjs.com...

[Kingpin2k]

yup indeed, I saw they were featured in, but I'm not sure if they are going to build it in it.

[ummdorian]

This solution writes to a new file, but the question asked about editing a file.