[SLL] multi-threading

[Robert Woodcock]

Below is the script I ended up with. I made a lot of stylistic changes so I
could read it better, err, I mean, so Paul couldn't, err, I mean...

(It would have been a rewrite one way or the other. :)

#!/bin/bash
IPLIST=/tmp/ips
RBLS="bl.spamcop.net dnsbl.sorbs.net no-more-funn.moensted.dk zen.spamhaus.org dnsbl.njabl.org dnsbl-3.uceprotect.net"
MAXLOOKUPS=6
OUTDIR=$(mktemp -td rblcheck.XXXXXX) || exit 1
STARTED=$OUTDIR/started
FINISHED=$OUTDIR/finished
touch $STARTED $FINISHED
cd $OUTDIR
(
# Generate list of queries
for IP in $(cat $IPLIST)
do
  echo $IP
  REVIP=$(echo $IP | sed 's/\([^.]*\)\.\([^.]*\)\.\([^.]*\)\.\([^.]*\)/\4.\3.\2.\1/')
  for RBL in $RBLS
  do
    echo $REVIP.$RBL
  done
done
) | (
# Run queries in parallel
while read QUERY
do
  while true
  do
    RUNNING=$(expr $(wc -l < $STARTED) - $(wc -l < $FINISHED))
    if [ $RUNNING -lt $MAXLOOKUPS ]; then
      break
    fi
    sleep 1
  done
  echo >> $STARTED
  (
    host $QUERY > $QUERY 2>&1
    echo >> $FINISHED
  ) &
  done
)                                                
# Wait for all lookups to finish
while [ $(wc -l < $FINISHED) -lt $(wc -l < $STARTED) ]
do
  sleep 1
done
# Display output
cat << ENDHEADER
 __ not listed    __ bl.spamcop.net
/   __ listed    /  __ dnsbl.sorbs.net
|  /            |  /   __ no-more-funn.moensted.dk
| |             | |  /   __ zen.spamhaus.org
0 1             | | |  /   __ dnsbl.njabl.org
                | | | |  /   __  dnsbl-3.uceprotect.net
                | | | | |  /                                    
  Check IP      | | | | | |  Total  Reverse Lookup
ENDHEADER
for IP in $(cat $IPLIST)
do
  printf "%-15s" $IP
  REVIP=$(echo $IP | sed 's/\([^.]*\)\.\([^.]*\)\.\([^.]*\)\.\([^.]*\)/\4.\3.\2.\1/')
  ONBLS=0
  BLNUM=0
  for RBL in $RBLS
  do
    RESULTFILE=$REVIP.$RBL
    if grep found $RESULTFILE >/dev/null; then
      # not found in DNS - not in blacklist
      echo -n " 0"
    else
      # no "not found" error - blacklisted
      echo -n " 1" # blacklisted
      ONBLS=$(expr $ONBLS + 1)
      COUNT[$BLNUM]=$(expr ${COUNT[$BLNUM]} + 1)
    fi
    BLNUM=$(expr $BLNUM + 1)
  done
  CLEANPTR=$(cut -d' ' -f5 $IP | egrep -v '(NXDOMAIN|alias|SERVFAIL|^for$)')
  echo -e " -- $ONBLS -- $CLEANPTR"
done
# generate column totals
echo -e "\n Column totals for each RBL, in order tested."
BLNUM=0
for RBL in $RBLS
do
  echo -e "${COUNT[$BLNUM]}\t${RBL}"
  BLNUM=$(expr $BLNUM + 1)
done
cd ..
rm -rf $OUTDIR

-- 
Robert Woodcock - rcw at blarg.net
"It's not based on any particular data point. We just wanted to choose a
really large number."
	-- A US Treasury spokeswoman on how the $700 billion bailout figure
           was arrived at