Be able to send stdout/stderr to both file and email

You can achieve this by forcing an email to be sent upon completion, and defining a custom $LSB_MAILPROG (set in lsf.conf) which will be called instead of sendmail.

Specifying -N in addition to -o/-e will force an email to be generated when the job completes.

$bsub -N -o out.txt -e out.err echo hello

Create a custom LSB_MAILPROG, for example:

#!/bin/bash
tmpfile=/tmp/$$.out
#The standard email message is buffered on stdin, so save that to the file first.

cat >$tmpfile

echo "Stdout from $LSB_OUTPUTFILE" >> $tmpfile

cat $LSB_OUTPUTFILE >> $tmpfile



echo "stderr from $LSB_ERRORFILE" >> $tmpfile

cat $LSB_ERRORFILE >> $tmpfile



/usr/lib/sendmail $1 $2 $3 $4 < $tmpfile

rm $tmpfile

The custom mailprog ultimately needs to call sendmail if you want the mail sent. Note, if the output or error are huge, you may clobber your mail system. So it it prudent to do some size checks, or maybe just head/tail the output.

The generated $tmpfile would be something like:

To: billmc

From: LSF <billmc>

Auto-submitted: auto-generated

X-Auto-Response-Suppress: OOF, AutoReply

Sender: LSF System <billmc@pf3>

Subject: Job 1454: <echo hello> in cluster <wsl> Done



Job <echo hello> was submitted from host <pf3> by user <billmc> in cluster <wsl> at Tue Apr  2 10:07:48 2024

Job was executed on host(s) <pf3>, in queue <normal>, as user <billmc> in cluster <wsl> at Tue Apr  2 10:07:48 2024

</home/billmc> was used as the home directory.

</home/billmc> was used as the working directory.

Started at Tue Apr  2 10:07:48 2024

Terminated at Tue Apr  2 10:07:52 2024

Results reported at Tue Apr  2 10:07:52 2024



Your job looked like:



------------------------------------------------------------

# LSBATCH: User input

echo hello

------------------------------------------------------------



Successfully completed.



Resource usage summary:



    CPU time :                                   0.02 sec.

    Max Memory :                                 -

    Average Memory :                             -

    Total Requested Memory :                     -

    Delta Memory :                               -

    Max Swap :                                   -

    Max Processes :                              -

    Max Threads :                                -

    Run time :                                   4 sec.

    Turnaround time :                            4 sec.



Read file <out.txt> for stdout output of this job.

Read file <out.err> for stderr output of this job.



Stdout from out.txt

hello

stderr from out.err