Bash: Guide to Bash IFS variable
The IFS is an acronym for Internal Field Separator or Input Field Separator. The $IFS is a special shell variable in Bash, ksh, sh, and POSIX. Let us see what $IFS is and why you need to use it while writing shell scripts under Linux and Unix.
The IFS (Internal Field Separator) is a special variable that defines the character or characters used to separate a pattern into tokens for some operations. The default value of IFS is a space, tab, and newline character ( \t\n), which means that by default, Bash uses these characters to separate words in a string.
What is IFS
- The IFS is a special shell variable.
- The IFS (Internal Field Separator) is used for word splitting after expansion and to split lines into words with the read builtin command .
- You can change the value of IFS as per your requirments.
- The default value is
<space><tab><newline>
- You can print it with the following command:
cat -etv <<<"$IFS"
- reset the Bash behavior to its default:
unset IFS
bash man page
The shell treats each character of IFS as a delimiter, and splits the results of the other expansions into words on these characters. If IFS is unset, or its value is exactly<space><tab><newline>
, the default, then sequences of This variable is used in a few different places. The semantics vary slightly, for example:
- In the read command, if multiple variable-name arguments are specified, IFS is used to split the line of input so that each variable gets a single field of the input. (The last variable gets all the remaining fields, if there are more fields than variables.)
- Also in the read command, any whitespace characters in IFS will be trimmed from the beginning and end of the input line, even when only one variable is given. In bash, however, specifying zero variable names to read suppresses IFS whitespace trimming.
- When performing WordSplitting on an unquoted expansion, IFS is used to split the value of the expansion into multiple words. (Each of these words may then undergo filename expansion.)
- When performing the
"$*"
or"${array[*]}"
expansion (*
not@
, and quoted), the first character of IFS is placed between the elements in order to construct the final output string. - Likewise, when doing
"${!prefix*}"
, the first character of IFS is placed between the variable names to make the output string. - IFS is used by
complete -W
under programmable completion.
Example 1
Create a text file called /tmp/domains.txt as follows:
mybluelinux.biz|2.54.1.1|/home/httpd|ftpuser1
mybluelinux.com|2.54.1.2|/home/httpd|ftpuser2
mybluelinux.org|2.54.1.3|/home/httpd|ftpuser3
Create a shell script as follows:
#!/bin/bash
file=/tmp/domains.txt
# set the Internal Field Separator to |
IFS='|'
while read -r domain ip webroot ftpusername
do
printf "*** Adding %s to httpd.conf...\n" $domain
printf "Setting virtual host using %s ip...\n" $ip
printf "DocumentRoot is set to %s\n" $webroot
printf "Adding ftp access for %s using %s ftp account...\n\n" $domain $ftpusername
done < "$file"
Output for this script is:
*** Adding mybluelinux.biz to httpd.conf...
Setting virtual host using 2.54.1.1 ip...
DocumentRoot is set to /home/httpd
Adding ftp access for mybluelinux.biz using ftpuser1 ftp account...
*** Adding mybluelinux.com to httpd.conf...
Setting virtual host using 2.54.1.2 ip...
DocumentRoot is set to /home/httpd
Adding ftp access for mybluelinux.com using ftpuser2 ftp account...
*** Adding mybluelinux.org to httpd.conf...
Setting virtual host using 2.54.1.3 ip...
DocumentRoot is set to /home/httpd
Adding ftp access for mybluelinux.org using ftpuser3 ftp account...
Where:
- The read command reads input from $file file.
- Each line of $file is broken into tokens with the help of $IFS.
- The value of IFS (|) are used as token delimiters or separator for each line.
- Each line is divided into four fields as domain, ip, webroot, and ftpusername.
- The while loop is used to read entier $file.
- The first token (Apache virtual hosting domain name) is saved to the actual variable called $domain.
- The second token (Apache ip address) is saved to the actual variable called $ip.
- The third token (Apache DocumentRoot) is saved to the actual variable called $webroot.
- The fourth token (FTP server username) is saved to the actual variable called $ftpusername.
IFS Effect On The Values of "$@" And "$*"
$@
and$*
are special command line arguments shell variables.- The
$@
holds list of all arguments passed to the script. - The
$*
holds list of all arguments passed to the script.
example:
#!/bin/bash
# ifsargs.sh - Cmd args - positional parameter demo
echo "Command-Line Arguments Demo"
echo "*** All args displayed using \$@ positional parameter ***"
echo $@
echo "*** All args displayed using \$* positional parameter ***"
echo $*
# start script with ./ifsargs.sh honda yamaha harley-davidson kawasaki
# Output:
Command-Line Arguments Demo
*** All args displayed using $@ positional parameter ***
honda yamaha harley-davidson kawasaki
*** All args displayed using $* positional parameter ***
honda yamaha harley-davidson kawasaki
- As you can see, the values of
$@
and$*
are same - However, the values of
"$@"
and"$*"
are different (note double quotes).
example:
#!/bin/bash
# ifsargs.sh - Cmd args - positional parameter demo
#### Set the IFS to | ####
IFS='|'
echo "Command-Line Arguments Demo"
echo "*** All args displayed using \$@ positional parameter ***"
echo "$@" #*** double quote added ***#
echo "*** All args displayed using \$* positional parameter ***"
echo "$*" #*** double quote added ***#
# start script with ./ifsargs.sh honda yamaha harley-davidson kawasaki
# Output:
Command-Line Arguments Demo
*** All args displayed using $@ positional parameter ***
honda yamaha harley-davidson kawasaki
*** All args displayed using $* positional parameter ***
honda|yamaha|harley-davidson|kawasaki
$@
expanded as "$1" "$2" "$3" ... "$n"$*
expanded as "$1y$2y$3y...$n", where y is the value of IFS variable i.e."$*"
is one long string and $IFS act as an separator or token delimiters.
Bash IFS with strings
There are special rules for handling whitespace characters in IFS, in any of the field-splitting situations above (the first three bullet points). Whitespace IFS characters at the beginning and end of a string are removed entirely (except in the special case noted above), and consecutive whitespace IFS characters inside a string are treated as a single delimiter. For example, consider the following:
#!/bin/bash
IFS=:
read -r user pwhash uid gid gecos home shell <<< 'statd:x:105:65534::/var/lib/nfs:/bin/false'
echo "=$user= =$pwhash= =$uid= =$gid= =$gecos= =$home= =$shell="
# output:
=statd= =x= =105= =65534= == =/var/lib/nfs= =/bin/false=
IFS=$' \t\n'
read -r one two three <<< ' 1 2 3'
echo "=$one= =$two= =$three="
# output:
=1= =2= =3=
In the first example, the gecos variable is assigned the empty string, which is the contents of the field between the two adjacent colons. The colons are not consolidated together; they are treated as separate delimiters. In the second example, the one variable gets the value 1, and the two variable gets the value 2. The leading whitespace is trimmed, and the internal whitespace is consolidated.
If IFS contains a mixture of whitespace and non-whitespace characters, it treats them differently. Any non-whitespace IFS character plus all adjacent IFS whitespace characters acts as a single field delimiter. (In addition, any sequence of one or more whitespace IFS characters also still counts.) For example:
#!/bin/bash
IFS=' ,'
var='this, that , the other'
printf '<%s> ' $var; echo
# output:
<this> <that> <the> <other>
The comma and space after this are treated as a field delimiter, The comma plus the multiple spaces around it after that are the second field delimiter. Finally, the single space after the is the final field delimiter, yielding a total of four words (fields).
More random examples:
#!/bin/bash
IFS=
read -r a b c <<< 'the plain gold ring'
echo "=$a="
# output:
=the plain gold ring=
IFS=$' \t\n'
read -r a b c <<< 'the plain gold ring'
echo "=$c="
# output:
=gold ring=
IFS=$' \t\n'
read -r a b c <<< 'the plain gold ring'
echo "=$a= =$b= =$c="
# output:
=the= =plain= =gold ring=
The first example above shows the lack of splitting when IFS is empty. The second shows the last variable-name given to a read command absorbing all the remaining words of input. The third shows that splitting and delimiter-consolidation are not performed on the remaining part of a line when assigning excess fields to the last variable.
#!/bin/bash
IFS=: read -r a b c <<< '1:2:::3::4'
echo "=$a= =$b= =$c="
# output:
=1= =2= =::3::4=
Here's another look at having more input fields than variables. Note that out of the three consecutive colons which follow field 2, precisely one colon was removed in order to terminate field 2. The remaining two colons, as well as two more colons later on, were all left untouched, and assigned to variable c verbatim.
unset IFS
IFS variable syntax
IFS=$'\n'
vs IFS='\n'
Words of the form $'string' are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard.
"\n"
(or '\n'
) is not the same as $'\n'
. The first string consists of the characters backslash and n. $'\n'
results in a single newline character.
IFS in bash Arrays
When performing WordSplitting on an unquoted expansion, IFS is used to split the value of the expansion into multiple words.
When performing the "$*"
or "${array[*]}"
expansion (*
not @
, and quoted), the first character of IFS is placed between the elements in order to construct the final output string.
Likewise, when doing "${!prefix*}"
, the first character of IFS is placed between the variable names to make the output string.
example with another IFS variable:
#!/bin/bash
# set IFS to '+'
IFS='+'
myArray=("1st item" "2nd item" "3rd item" "4th item")
printf 'Word -> %s\n' ${myArray[@]} # output is array items because first char in $IFS not contain space
# output:
Word -> 1st item
Word -> 2nd item
Word -> 3rd item
Word -> 4th item
myArray=("1st item" "2nd item" "3rd item" "4th item")
printf 'Word -> %s\n' "${myArray[*]}" # use the full array as one word with first char in $IFS as delimeter
# output:
Word -> 1st item+2nd item+3rd item+4th item
myArray=("1st item" "2nd item" "3rd item" "4th item")
printf 'Word -> %s\n' "${myArray[@]}" # use arrays entries, because array variable is quoted
# output:
Word -> 1st item
Word -> 2nd item
Word -> 3rd item
Word -> 4th item
# set IFS to 'e' char
IFS='e'
myArray=("1st item" "2nd item" "3rd item" "4th item")
printf 'Word -> %s\n' ${myArray[@]} # output is array items with 'e' char as delimeter
# output:
Word -> 1st it
Word -> m
Word -> 2nd it
Word -> m
Word -> 3rd it
Word -> m
Word -> 4th it
Word -> m
# set IFS to default value
unset IFS
"${myArray[@]}"