strtok()

Purpose

strtok() splits a string into tokens, one token per call. Each token is a run of characters that sits between delimiter characters. Unlike explode(), it does not return an array. It remembers its position inside the string between calls, so you fetch the pieces one at a time in a loop.

The function is handy for simple scanning jobs: splitting words on any whitespace, reading a short delimited list, or pulling pieces out of a string when you do not want empty segments.

Signature

Parameters and defaults

• $string: the text to split. Pass it only on the first call, which starts a new tokenizing session.
• $token: a set of delimiter characters. It is not a delimiter substring. Passing " \n\t" means a space, a newline or a tab each end a token.

Neither parameter has a default value. In the one-argument form, the single argument is the delimiter set and the function continues from where the previous call stopped. The manual's changelog notes that since PHP 8.3.0 an E_WARNING is emitted when $token is not provided.

Return value and failure cases

strtok() returns the next token as a string, or false when no tokens remain. It does not throw for ordinary input. Calling it with a string that has no tokens, such as one made only of delimiters, gives false right away.

The manual warns that the result must be tested with the strict operator === or !==. A token such as "0" is a valid string but is falsy in a loose comparison, so while ($tok) can stop too early.

Example 1: walk through words

The first call passes the string. Every later call passes only the delimiters.

<?php
$string = "This is\tan example\nstring";
$tok = strtok($string, " \n\t");
while ($tok !== false) {
    echo "Word={$tok}\n";
    $tok = strtok(" \n\t");
}

Word=This
Word=is
Word=an
Word=example
Word=string

Example 2: strtok() versus explode()

The key behaviour is that consecutive delimiters count as one, and delimiters at the start or end of the string are ignored. explode() keeps every empty piece.

<?php
$string = ";aaa;;bbb;";

$parts = [];
$tok = strtok($string, ";");
while ($tok !== false) {
    $parts[] = $tok;
    $tok = strtok(";");
}
echo json_encode($parts), "\n";

$parts = explode(";", $string);
echo json_encode($parts), "\n";

["aaa","bbb"]
["","aaa","","bbb",""]

Example 3: change delimiters between calls

The delimiter set may differ on every call. Here the string alternates between keys and values, so the loop asks for a key with = and a value with ;.

<?php
$pairs = [];
$key = strtok("a=1;b=2;c=3", "=");
while ($key !== false) {
    $value = strtok(";");
    $pairs[$key] = $value;
    $key = strtok("=");
}
echo json_encode($pairs, JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES), "\n";

{"a":"1","b":"2","c":"3"}

Edge cases

• Empty tokens are skipped. You cannot tell from the results that two delimiters were adjacent. If empty fields matter, such as in CSV-like data, use explode() or a real parser instead.
• Leading delimiters vanish. Tokenizing "/something" with "/" gives "something" first, then false. There is no empty first token.
• Hidden state. The current position is stored internally between calls. Calling strtok() with a new string restarts the process and discards the old position.
• Delimiters are single characters. strtok($s, "ab") splits on a and on b, not on the text "ab".
• Falsy tokens. A token "0" is returned like any other. Only a strict comparison with false ends the loop safely.

Common mistakes

1. Writing while ($tok) instead of while ($tok !== false), which stops at a "0" token.
2. Passing the full string on every call, which restarts tokenizing and returns the first token forever, creating an infinite loop.
3. Expecting a multi-character delimiter such as ", " to be matched as a unit. Each character in it is a separate delimiter.
4. Expecting empty fields to be preserved, as explode() does.
5. Interleaving two tokenizing jobs. Because the position is shared, starting a second string in the middle of a loop, for example inside a helper function, breaks the first loop. Finish one pass before beginning another.
6. Omitting the delimiter argument on a later call. Always supply $token.

Related functions

• explode() splits by a whole delimiter string and returns an array that keeps empty pieces.

Recap

strtok() hands out tokens one at a time. Start with strtok($string, $delimiters), continue with strtok($delimiters), and stop when the result is false, tested with !==. Remember that delimiters are a set of characters, that runs of delimiters collapse, and that the position is shared state. Use explode() when you need empty fields kept.