Cycle through file contents in Bash

Posted by me1000 on Wed, 11 Dec 2019 03:03:43 +0100

How to use Bash Traverse each line of the text file?

Use this script:

echo "Start!"
for p in (peptides.txt)
do
    echo "${p}"
done

I get the following output on the screen:

Start!
./runPep.sh: line 3: syntax error near unexpected token `('
./runPep.sh: line 3: `for p in (peptides.txt)'

(later I wanted to do something more complicated with $p than just output to the screen.)

The environment variable SHELL is (from env):

SHELL=/bin/bash

/bin/bash --version output:

GNU bash, version 3.1.17(1)-release (x86_64-suse-linux-gnu)
Copyright (C) 2005 Free Software Foundation, Inc.

cat /proc/version output:

Linux version 2.6.18.2-34-default (geeko@buildhost) (gcc version 4.1.2 20061115 (prerelease) (SUSE Linux)) #1 SMP Mon Nov 27 11:46:27 UTC 2006

The peptide file.txt file contains:

RKEKNVQ
IPKKLLQK
QYFHQLEKMNVK
IPKKLLQK
GDLSTALEVAIDCYEK
QYFHQLEKMNVKIPENIYR
RKEKNVQ
VLAKHGKLQDAIN
ILGFMK
LEDVALQILL

#1 building

cat peptides.txt | while read line 
do
   # do something with $line here
done

And single line variants:

cat peptides.txt | while read line; do something_with_$line_here; done

If there is no trailing wrap, these options skip the last line of the file.

You can avoid this by:

cat peptides.txt | while read line || [[ -n $line ]];
do
   # do something with $line here
done

#2 building

One way is to:

while read p; do
  echo "$p"
done <peptides.txt

As noted in the comments, the side effects of this are trimming leading whitespace, explaining the backslash sequence, and skipping the last line if the last line is missing a newline character. If you have these problems, you can do the following:

while IFS="" read -r p || [ -n "$p" ]
do
  printf '%s\n' "$p"
done < peptides.txt

If The circulator can be read from the standard input Exception, you can use other file descriptors to open the file:

while read -u 10 p; do
  ...
done 10<peptides.txt

Here, 10 is just an arbitrary number (different from 0, 1, 2).

#3 building

Option 1a: While loop: one line at a time: input redirection

#!/bin/bash
filename='peptides.txt'
echo Start
while read p; do 
    echo $p
done < $filename

Option 1b: While loop: one line at a time:
Open the file and read from the file descriptor (in this case, file descriptor 4).

#!/bin/bash
filename='peptides.txt'
exec 4<$filename
echo Start
while read -u4 p ; do
    echo $p
done

#4 building

This is no better than the other answers, but it's another way to do your work in a file without spaces (see note). I find that I often need a line to mine the list in a text file without the extra steps of using a separate script file.

for word in $(cat peptides.txt); do echo $word; done

This format allows me to put everything on one command line. Change the "echo $word" section to what you want, and you can issue multiple semicolon delimited commands. The following example uses the contents of the file as parameters to two other scripts you might write.

for word in $(cat peptides.txt); do cmd_a.sh $word; cmd_b.py $word; done

Or, if you plan to use it as a stream editor (learn sed), dump the output to another file, as shown below.

for word in $(cat peptides.txt); do cmd_a.sh $word; cmd_b.py $word; done > outfile.txt

I used the above because I used a text file with only one word per line. (see note) if you have spaces that you don't want to split words / lines, it can get ugly, but the same command can still work as follows:

OLDIFS=$IFS; IFS=$'\n'; for line in $(cat peptides.txt); do cmd_a.sh $line; cmd_b.py $line; done > outfile.txt; IFS=$OLDIFS

This simply tells the shell to split only on line breaks, not spaces, and then return the environment to its previous state. At this point, you may want to consider putting it all in a shell script, rather than compressing it all into one line.

Good luck!

#5 building

#!/bin/bash
#
# Change the file name from "test" to desired input file 
# (The comments in bash are prefixed with #'s)
for x in $(cat test.txt)
do
    echo $x
done

Topics: shell Linux