Have Geek Brain

Processing a tab delimited file using shell script processing

normally I would use Python / Perl for this procedure, but I find myself (for political reasons) to do it with the bash shell.

I have a large tab delimited file with six columns and the second column is integers. I need to install a script solution that will check that the file actually has six columns and that the second column is indeed an integer. I guess I will need to use sed / awk here. The problem is that I am not familiar with sed / awk. Any advice would be appreciated.

Many thanks! Lilly

+2


a source to share


4 answers


Ok, you can tell directly awk

what the field separator is (-F option). Inside the awk

script, you can specify how many fields are present in each record using the NF variable.

Oh, and you can check the second field with regex. All of this might look something like this:



awk < thefile -F\\t '
{ if (NF != 6 || $2 ~ /[^0123456789]/) print "Format error, line " NR; }
'

This is probably close, but I need to check the regex because the Linux regex syntax variable is so crazy. (edited since grrrr)

+2


a source


simpleton:

BEGIN {
  FS="\t"
}

(NF != 6) || ($2 != int($2)) {
  exit 1
}


Call the following:

if awk -f colcheck.awk somefile
then
  # is valid
else
  # is not valid
fi
+3


a source


here's how to do it with awk

awk 'NF!=6||$2+0!=$2{print "error"}' file
+2


a source


Pure Bash:

infile='column6.dat'
lno=0

while read -a line ; do
  ((lno++))
  if [ ${#line[@]} -ne 6 ] ; then
    echo -e "line $lno has ${#line[@]} elements"
  fi
  if ! [[  ${line[1]} =~ ^[0-9]+$ ]] ; then
    echo -e "line $lno column  2 : not an integer"
  fi
done < "$infile"

Possible way out:

line 19 has 5 elements
line 36 column  2 : not an integer
line 38 column  2 : not an integer
line 51 has 3 elements
+2


a source






More articles:


All Articles