cmd.exe: for /f "tokens=…"

This page tries to demonstrate how the for /f tokens=… option influences the splitting of text into tokens and assigning them to variables.

A series of seven words (one two three four five six seven) is assigned to the variable %text%.

The for /f is then applied on this variable. The tokens=… option assigns the specified tokens to the %%A and following variables (%%B, %%C …).

In order to show which value was assigned to which variable, the variables A through E are printed after splitting %text% into tokens.

Because the for /f … command is in a batch file the variables %%A and so on need to have two percent signs. If the command were entered directly into the cmd.exe console, prefixing them with one % sign would be sufficient.

tokens=*

With tokens=*, all tokens, that is: the entire text, is assigned to one variable.

@set text=one   two  three four five    six seven

@for /f "tokens=*" %%A in ("%text%") do @(
  @echo A = %%A
  @echo B = %%B
  @echo C = %%C
  @echo D = %%D
  @echo E = %%E
)

Github repository about-cmd.exe, path: /commands/for/f/tokens/star.bat

The script prints:

A = one   two  three four five    six seven
B = %B
C = %C
D = %D
E = %E

tokens=3*

With tokens=3*, the first two tokens are skipped, the third token is assigned to the first variable and the rest to the second variable:

@set text=one   two  three four five    six seven

@for /f "tokens=3*" %%A in ("%text%") do @(
  @echo A = %%A
  @echo B = %%B
  @echo C = %%C
  @echo D = %%D
  @echo E = %%E
)

Github repository about-cmd.exe, path: /commands/for/f/tokens/3star.bat

A = three
B = four five    six seven
C = %C
D = %D
E = %E

tokens=1-4*

With tokens=1-4*, the first four tokens are assigned to the first four variables (%%A through %%D) and the rest is assigned to the next (in this case: the fifth) variable:

@set text=one   two  three four five    six seven

@for /f "tokens=1-4*" %%A in ("%text%") do @(
  @echo A = %%A
  @echo B = %%B
  @echo C = %%C
  @echo D = %%D
  @echo E = %%E
)

Github repository about-cmd.exe, path: /commands/for/f/tokens/1-4star.bat

A = one
B = two
C = three
D = four
E = five    six seven

tokens=1-4,*

tokens=1-4,* behaves exactly as tokens=1-4* does. So, the comma in front of the asterisk does not change any meaning.

@set text=one   two  three four five    six seven

@for /f "tokens=1-4*" %%A in ("%text%") do @(
  @echo A = %%A
  @echo B = %%B
  @echo C = %%C
  @echo D = %%D
  @echo E = %%E
)

Github repository about-cmd.exe, path: /commands/for/f/tokens/1-4commaStar.bat

A = one
B = two
C = three
D = four
E = five    six seven

tokens=2,4,5

With tokens=2,4,5, only the second, forth and fifth tokens are assigned. The rest is omitted.

A = two
B = four
C = five
D = %D
E = %E

tokens=2,4,5,*

tokens=2,4,5, * again assigns the second, fourth and fifth token. The star assigns everything after the fifth token to the next (here: the fourth) variable:

@set text=one   two  three four five    six seven

@for /f "tokens=2,4,5,*" %%A in ("%text%") do @(
  @echo A = %%A
  @echo B = %%B
  @echo C = %%C
  @echo D = %%D
  @echo E = %%E
)

Github repository about-cmd.exe, path: /commands/for/f/tokens/2_4_5_star.bat

A = two
B = four
C = five
D = six seven
E = %E