- PHP Generator - reading file content
- Read file using a loop
- Read file using
a generator
- Generator
- The end
In PHP, to fetch content from a file, a common implementation is to loop its content and store them in a temporary variable. In today's tutorial, we want to introduce a solution by using the
generator.
Read file using a loopNormal we will load the file content to memory and read its content using a loop. $m1 = memory_get_peak_usage();
foreach (file('lorem.txt') as $l);
$m2 = memory_get_peak_usage();
echo $m2 - $m1."\n";
// Output
1542736
Read
file using a generatorWe can do the same using a generator function. Inside the function, we open the file and yield its content line by line: function file_lines($filename) {
$file = fopen($filename, 'r');
while (($line = fgets($file)) !== false) {
yield $line;
}
fclose($file);
}
$m1 = memory_get_peak_usage();
foreach (file_lines('lorem.txt') as $l);
$m2 = memory_get_peak_usage();
echo $m2 - $m1."\n";
// Ouput
0
GeneratorAs we can see, when using a generator, we have greatly reduced memory usage. The reason is that when we yield results from a generator function, it saves its
current state until it is called next time. The endHopefully this simple tutorial helped you with your development. (PHP 5 >= 5.5.0, PHP 7, PHP 8) Generators provide an easy way to implement simple iterators without the overhead or complexity of implementing a class that implements the Iterator interface. A generator allows you to write code that uses
foreach to iterate over a set of data without needing to build an array in memory, which may cause you to exceed a memory limit, or require a considerable amount of processing time to generate. Instead, you can write a generator function, which is the same as a normal function, except that instead of returning once, a generator can yield as many times as it needs to in order to provide the values to be iterated over. A simple example of this is to reimplement the range() function as a generator. The standard
range() function has to generate an array with every value in it and return it, which can result in large arrays: for example, calling range(0, 1000000) will result in well over 100 MB of memory being used. As an alternative, we can implement an xrange() generator, which will only ever need enough memory to create an
Iterator object and track the current state of the generator internally, which turns out to be less than 1 kilobyte. Example #1 Implementing range() as a generator
<?php function xrange($start, $limit, $step = 1) { if ($start <= $limit) { if ($step <= 0) { throw new LogicException('Step must be positive'); }
for (
$i = $start; $i <= $limit; $i += $step) { yield $i; } } else { if ($step >= 0) { throw new LogicException('Step must be negative'); } for ( $i = $start; $i >= $limit; $i += $step) { yield $i; } } }/* * Note that both range() and xrange() result in the same * output below. */echo 'Single digit odd numbers from range(): '; foreach (range(1, 9, 2) as $number) { echo "$number "; } echo "\n";echo 'Single digit odd numbers from xrange(): '; foreach (xrange(1, 9, 2) as $number) { echo "$number "; } ?>
The above example will output: Single digit odd numbers from range(): 1 3 5 7 9
Single digit odd numbers from xrange(): 1 3 5 7 9
Generator objects When a generator function is called, a new object of the internal Generator class is returned. This object implements the Iterator interface in much the same
way as a forward-only iterator object would, and provides methods that can be called to manipulate the state of the generator, including sending values to and returning values from it. montoriusz at gmail dot com ¶
6 years ago
Bear in mind that execution of a generator function is postponed until iteration over its result (the Generator object) begins. This might confuse one if the result of a generator is assigned to a variable instead of immediate iteration.
<?php
$some_state
= 'initial';function gen() { global $some_state; echo "gen() execution start\n"; $some_state = "changed"; yield 1; yield 2; }function peek_state() { global $some_state; echo "\$some_state = $some_state\n"; }echo "calling gen()...\n"; $result = gen(); echo "gen() was called\n";peek_state();echo "iterating...\n"; foreach ($result as $val) { echo "iteration: $val\n"; peek_state(); }?>
If you need to perform some action when the function is called and before the result is used, you'll have to wrap your generator in another function.<?php /** * @return Generator */ function some_generator() { global $some_state;$some_state = "changed"; return gen(); } ?>
info at boukeversteegh dot nl ¶ 6 years ago
Here's how to detect loop breaks, and how to handle or cleanup after an interruption.
<?php function generator() { $complete = false; try {
while ((
$result = some_function())) { yield $result; } $complete = true; } finally { if (! $complete) { // cleanup when loop breaks } else { // cleanup when loop completes } }// Do something only after loop completes } ?>
lubaev ¶ 8 years ago
Abstract test. <?php
$start_time
=microtime(true); $array = array(); $result = ''; for($count=1000000; $count--;) { $array[]=$count/2; } foreach($array as $val) { $val += 145.56; $result .= $val; } $end_time=microtime(true);echo "time: ", bcsub($end_time, $start_time, 4), "\n"; echo "memory (byte): ", memory_get_peak_usage(true), "\n";?>
<?php$start_time =microtime(true); $result = ''; function it() { for($count=1000000; $count--;) { yield $count/2; } } foreach(it() as $val) { $val += 145.56; $result .= $val; } $end_time=microtime(true);echo "time: ", bcsub($end_time, $start_time, 4), "\n"; echo "memory (byte): ", memory_get_peak_usage(true), "\n";?> Result: ---------------------------------- | time | memory, mb | ---------------------------------- | not gen | 2.1216 | 89.25 | |--------------------------------- | with gen | 6.1963 | 8.75 | |--------------------------------- | diff | < 192% | > 90% | ----------------------------------
dc at libertyskull dot
com ¶ 8 years ago
Same example, different results:
---------------------------------- | time | memory, mb | ---------------------------------- | not gen | 0.7589 | 146.75 | |--------------------------------- | with gen | 0.7469 | 8.75 | |---------------------------------
Time in results varying from 6.5 to 7.8 on both examples. So no real drawbacks concerning processing speed.
youssefbenhssaien at gmail dot com ¶ 5
years ago
A simple function to parse an ini configuration file <?php function parse_ini($file_path){ if(!file_exists($file_path)){ throw new Exception("File not exists ${file_path}"); } $text = fopen($file_path, 'r'); while($line=fgets($text)){ list($key, $param) = explode('=', $line); yield $key => $param; } } ?> //Usage : parse_ini('param.ini') // returns Generator Object //Usage : iterator_to_array(parse_ini('param.ini')); // returns an array
Anonymous ¶ 3 years ago
Same example, different results:
---------------------------------- | time | memory, mb | ---------------------------------- | not gen | 0.7589 | 146.75 | |--------------------------------- | with gen | 0.7469 | 8.75 | |---------------------------------
Time in results varying from 6.5 to 7.8 on both exassmples. So no real drawbacks concerning processing speed.
|