TL;DR

To avoid getting repeated screen messages from Sherpa commands (with a red/pink background), change the behavior of the Sherpa logging instance with the following code:

{.python}
from sherpa.astro.ui import *
import logging
logging.getLogger('sherpa').propagate = False

Read more for finding out what this does, and why.

Strange output from Sherpa commands

Sherpa uses the Python logging framework for outputting much of its information. This lets you do fancy things - such as mute the output in a script or direct to a separate file for later analysis. However, it appears to have some strange consequences for use in IPython notebooks, as we show below.

In [1]:
from sherpa.astro.ui import *

In this notebook directory I have placed a few files; for instance a source spectrum, along with the response files (ARF and RMF), but no background file.

In [2]:
ls src*
src.arf  src.pi   src.rmf

Loading in an X-ray spectrum into Sherpa

We can load in this spectrum; note that the lines in red/pink below are from the Python logging framework.

In [3]:
load_pha('src.pi')
read ARF file src.arf
INFO:sherpa.astro.io:read ARF file src.arf
read RMF file src.rmf
INFO:sherpa.astro.io:read RMF file src.rmf

Fitting the data

The fit and error commands - conf and covar - also use logging, hence the red/pink lines repeating information.

In [4]:
set_source(xsphabs.gal * xsapec.src)
gal.nh = 0.05
freeze(gal.nh)
thaw(src.abundanc)
fit()
Dataset               = 1
Method                = levmar
Statistic             = chi2gehrels
Initial fit statistic = 1.51469e+11
Final fit statistic   = 536.896 at function evaluation 201
Data points           = 1024
Degrees of freedom    = 1021
Probability [Q-value] = 1
Reduced statistic     = 0.525853
Change in statistic   = 1.51469e+11
   src.kT         5.06954     
   src.Abundanc   0.000262452 
   src.norm       0.000249864 
INFO:sherpa.astro.ui.utils:Dataset               = 1
Method                = levmar
Statistic             = chi2gehrels
Initial fit statistic = 1.51469e+11
Final fit statistic   = 536.896 at function evaluation 201
Data points           = 1024
Degrees of freedom    = 1021
Probability [Q-value] = 1
Reduced statistic     = 0.525853
Change in statistic   = 1.51469e+11
   src.kT         5.06954     
   src.Abundanc   0.000262452 
   src.norm       0.000249864 
In [5]:
covar()
WARNING: hard minimum hit for parameter src.kT
WARNING:sherpa.fit:hard minimum hit for parameter src.kT
WARNING: hard maximum hit for parameter src.kT
WARNING:sherpa.fit:hard maximum hit for parameter src.kT
WARNING: hard minimum hit for parameter src.Abundanc
WARNING:sherpa.fit:hard minimum hit for parameter src.Abundanc
WARNING: hard minimum hit for parameter src.norm
WARNING:sherpa.fit:hard minimum hit for parameter src.norm
WARNING: hard maximum hit for parameter src.norm
WARNING:sherpa.fit:hard maximum hit for parameter src.norm
Dataset               = 1
Confidence Method     = covariance
Iterative Fit Method  = None
Fitting Method        = levmar
Statistic             = chi2gehrels
covariance 1-sigma (68.2689%) bounds:
   Param            Best-Fit  Lower Bound  Upper Bound
   -----            --------  -----------  -----------
   src.kT            5.06954        -----        -----
   src.Abundanc  0.000262452        -----    0.0668767
   src.norm      0.000249864        -----        -----
INFO:sherpa.ui.utils:Dataset               = 1
Confidence Method     = covariance
Iterative Fit Method  = None
Fitting Method        = levmar
Statistic             = chi2gehrels
covariance 1-sigma (68.2689%) bounds:
   Param            Best-Fit  Lower Bound  Upper Bound
   -----            --------  -----------  -----------
   src.kT            5.06954        -----        -----
   src.Abundanc  0.000262452        -----    0.0668767
   src.norm      0.000249864        -----        -----
In [6]:
conf()
Dataset               = 1
Confidence Method     = confidence
Iterative Fit Method  = None
Fitting Method        = levmar
Statistic             = chi2gehrels
confidence 1-sigma (68.2689%) bounds:
   Param            Best-Fit  Lower Bound  Upper Bound
   -----            --------  -----------  -----------
   src.kT            5.06954     -1.37989       2.5168
   src.Abundanc  0.000262452        -----     0.217209
   src.norm      0.000249864 -1.89861e-05  1.80565e-05
src.Abundanc lower bound:	-----
src.norm lower bound:	-1.89861e-05
src.kT lower bound:	-1.37989
src.Abundanc upper bound:	0.217209
src.norm upper bound:	1.80565e-05
src.kT upper bound:	2.5168
INFO:sherpa.ui.utils:Dataset               = 1
Confidence Method     = confidence
Iterative Fit Method  = None
Fitting Method        = levmar
Statistic             = chi2gehrels
confidence 1-sigma (68.2689%) bounds:
   Param            Best-Fit  Lower Bound  Upper Bound
   -----            --------  -----------  -----------
   src.kT            5.06954     -1.37989       2.5168
   src.Abundanc  0.000262452        -----     0.217209
   src.norm      0.000249864 -1.89861e-05  1.80565e-05
INFO:sherpa:src.Abundanc lower bound:	-----
INFO:sherpa:src.norm lower bound:	-1.89861e-05
INFO:sherpa:src.kT lower bound:	-1.37989
INFO:sherpa:src.Abundanc upper bound:	0.217209
INFO:sherpa:src.norm upper bound:	1.80565e-05
INFO:sherpa:src.kT upper bound:	2.5168

Hiding the logging messages

Can we turn off the logging? The initial approach I tried was to hide informational-level messages (you can see that the messages in red/pink are labelled INFO:) using the Sherpa logger; that is the following

{.python}
import logging
logging.getLogger('sherpa').setLevel(logging.WARN)

However, this just hid all the screen output - e.g. load_pha and fit would display no screen messages. I then tried to adjust the root logger by removing its handler - e.g.

{.python}
import logging
logger = logging.getLogger()
logger.removeHandler(logger.handlers[0])

but this seems a bit drastic.

I also tried setting --log-level=WARN as a command-line option when running ipython, but apart from that being fragile - since you need to remember to do it - it doesn't work, presumably for the same reason as changing the level of the root logger to logging.WARN (not shown) also fails; I am guessing that since the sherpa instance has accepted the message the level check by the root logger is ignored.

So far, the best solution appears to be to stop the sherpa logger from propogating its messages (although I am wondering about adding some filter to the handler of the root logger to just ignore INFO-level messages, since we then still get the red/pink background for warnings, at the expense of duplicated screen output).

In [7]:
import logging
logging.getLogger('sherpa').propagate = False
In [8]:
load_pha('src.pi')
read ARF file src.arf
read RMF file src.rmf

Success!

In [9]:
fit()
Dataset               = 1
Method                = levmar
Statistic             = chi2gehrels
Initial fit statistic = 536.896
Final fit statistic   = 536.895 at function evaluation 154
Data points           = 1024
Degrees of freedom    = 1021
Probability [Q-value] = 1
Reduced statistic     = 0.525852
Change in statistic   = 0.00129407
   src.kT         5.07946     
   src.Abundanc   0           
   src.norm       0.000250997 
WARNING: parameter value src.Abundanc is at its minimum boundary 0.0
In [10]:
covar()
WARNING: hard minimum hit for parameter src.kT
WARNING: hard maximum hit for parameter src.kT
WARNING: hard minimum hit for parameter src.Abundanc
WARNING: hard minimum hit for parameter src.norm
WARNING: hard maximum hit for parameter src.norm
Dataset               = 1
Confidence Method     = covariance
Iterative Fit Method  = None
Fitting Method        = levmar
Statistic             = chi2gehrels
covariance 1-sigma (68.2689%) bounds:
   Param            Best-Fit  Lower Bound  Upper Bound
   -----            --------  -----------  -----------
   src.kT            5.07946        -----        -----
   src.Abundanc            0        -----    0.0555689
   src.norm      0.000250997        -----        -----
In [11]:
conf()
Dataset               = 1
Confidence Method     = confidence
Iterative Fit Method  = None
Fitting Method        = levmar
Statistic             = chi2gehrels
confidence 1-sigma (68.2689%) bounds:
   Param            Best-Fit  Lower Bound  Upper Bound
   -----            --------  -----------  -----------
   src.kT            5.07946     -1.39265      2.47115
   src.Abundanc            0        -----     0.216644
   src.norm      0.000250997 -2.00667e-05  1.69129e-05
src.Abundanc lower bound:	-----
src.norm lower bound:	-2.00667e-05
src.kT lower bound:	-1.39265
src.Abundanc upper bound:	0.216644
src.norm upper bound:	1.69129e-05
src.kT upper bound:	2.47115

It is interesting that the conf output shows the table before the individual lines, unlike at the command line. This can also be seen in the earlier call to conf but it's more obvious here.