Notice the addition of
algorithm to detect the end of the input string. When strtok encounters the end of
input, it returns FALSE, so your first inclination might be to test for FALSE in the for
loop. Recall that an empty string is considered equivalent to FALSE. If two separators
follow each other, strtok will return an empty string, as you'd expect. Since we don't
want to stop tokenizing at the first repeated separator, we place a token at the end that we
know won't appear in the input. If we're worried about people purposely putting
in the input string, we could strip it out first, but this isn't something that will be typed by
accident. Since there's no security risk to the tokenizing ending too soon, I prefer to let
hackers get invalid results.
The strtok function is useful only in the most simple and structured situations. An
example might be reading a tab-delimited text file. The algorithm might be to read a line
from a file, pulling each token from the line using the tab character, then continuing by
getting the next line from the file.
Regular Expressions
Fortunately, PHP offers something more powerful than the strtok function: regular
expressions. Written in a language of their own, regular expressions describe patterns that
are compared to strings. The PHP source code includes an implementation of regular
expressions that conform to the POSIX 1003.2 standard. This standard allows for
expressions of an older style but encourages a modern style that I will describe. All the
regular expression functions are described in Chapter 9.
In 1999, Andrei Zmievski added support for regular expressions that follow the style of
Perl. They offer two advantages over PHP native regular expressions. They make it easier
to copy an expression from a Perl script, and they take less time to execute.
It is beyond the scope of this text to examine regular expressions in depth. It is a subject
worthy of a book itself. I will explain the basics, as well as demonstrate the various PHP
functions that use regular expressions. An excellent resource for learning more about
regular expressions is Chapter 2 of Ellie Quigley's UNIX Shells by Example. If you are
interested in PERL-style regular expressions, first read the official documentation for
Perl, compiled and edited by Tom Christiansen <http://www.perl. com/CPAN-
local/doc/manual/html/pod/perlre.html>. You will then need to read the
documentation at the PHP site itself that lists the differences between Perl and the PHP
implementation <http://www.php. net/manual/pcre.pattern.syntax.php3>.
Defining Regular Expressions