Setting up daq and actions on problems

Setting up daq program and actions on problems

Settings up daq paramters
On problems with daq ...
Offline analysis
Analysis using FLT log files
Tn/Tv/T_shape settings
DAQ parameters
DAQ parameter files
Handling non-typical situations
Cooling strategy
ASAS photmetry/astrometry

1. Setting up daq parameters.

   Every night catalog for data is created ( script prepdaq.sh in crontab ) , it is created in directory /data2/results on pi2, with name YYYYMMDD , for example :

             /data2/results/20050215

All daq configuration files *.cfg are copied there, also satellite DB is copied there.
In order to change some parameters file ccd.cfg should be edited. It is possible to set up specific settings ( like temperature, gain etc ... ) for cameras by editing files device0.cfg (k2a) and device1.cfg (k2b).
In order to change other parameters not related to camera, but more to algorithms ccd_pipeline0.cfg (k2a) and ccd_pipeline1.cfg (k2b) should be modified
There are also other global settings cfg files :

       dynamic.cfg - in this file settings changed during system operation are stored ( like Tn at full moon etc )
       device.cfg - common device settings ( same for both camers ) , if some settings should be different use device0.cfg and device1.cfg
       cam_and_lens.cfg - settings of camera optic information , only description written to FITS headers
       obs_site_LCO.cfg - coordinates of observation location and description
      daqstat.cfg - also created in real time , contains information about current state of daq program

Files dynamic.cfg and daqstat.cfg are created in order to have all nesessary information about current state of daq program in case of daq re-start, but this can sometimes lead to problems in case
some piman commands were already executed meanwhile , in such situation just after killing ccddouble program in order to restart daq this two files should be removed to ignore any previous
state of daq.

Other files used :
     satelitesdb.tle - database of satellites created daily ( script prepdaq.sh from crontab at 23 LCO TIME )
     satelitesdb.qth - settings of obsevation location for sattelites checking module ( longitude and latitude of observation site )

In order to overwrite some settings for example to specify that shutter time is 5 sec (not default 10 sec) , no the default one , edit files in directory :
      /opt/pi/dev/pisys/daq/config/custom/
in this case it will be :
     /opt/pi/dev/pisys/daq/config/custom/device.cfg
put the following line there :
   CCD_DRIVER_SHUTTER_TIME_IN_SEC=5
this change will allow you not to change this setting every night it will be used until this line will be removed.

2. On problems with daq ...

- in case daq is hang or errors of driver ( /dev/usb/pikam0 or /dev/usb/pikam1 ) , occure one should do the following actions :

              ps -ef | grep ccddouble
              take PID of ccddouble process and run :
              kill PID

- in case after re-loading driver and daq restart it is in strange state ( most likely to occure when restart was done during scan ) , one should go to directory with frames and cfg files for current night :

              cd /data2/results/YYYYMMDD
              ps -ef | grep ccddouble
              kill PID
              rm -f daqstat.cfg dynamic.cfg

   removing of this two cfg files should restore default daq state, it should be done before ccddouble program is restart - about 90 sec after killing it.

- in any case after daq re-start in pishell the following command should be executed :

             internal 0 send_pos_to_daq

3. Offline analysis

NOTE : always remember that some settings are overwritten in files ccd_pipeline0.cfg and ccd_pipeline1.cfg and must be changed
in these files not just in ccd.cfg
general rules

For double camera in coincydence use cfg files from $SRCDIR/ccd/cfg/WZORY_CCD_CFG/doubleCAM_Real/2K2K_2K2K , for
single camera with confirmation on next frames use $SRCDIR/ccd/cfg/WZORY_CCD_CFG/doubleCAM_Real/2K2K/
For other cfg files types location check here

In order to run offline analysis file offline.cfg ( in $SRCDIR/ccd/cfg/WZORY_CCD_CFG/doubleCAM_Real/2K2K_2K2K ) should be included at
the very begining of the ccd.cfg file and the following parameters should be correctly set :

            CCD_FULL_FRAMES_SAMPLE_DIR= path to directory where frames ( fit or fitc files ) are stored

   In ccd_pipeline0.cfg and ccd_pipeline1.cfg :

          CCD_DARK_FRAME_FILE=                   path to dark frame file
Prepare list of file frames_list_ccd1 and frames_list_ccd2

After this changes ccddouble or Nparamstest2C can be run and will do analysis using frame files from disk
To be sure set that no type of OBJECT is skiped , set empty paramter :
          CCD_SKIP_IF_OBJECT_MATCHES=
make sure that correct location file is loaded - look for obs_site in cfg file and load obs_site_brwinow.cfg for Brwinow test location , or
obs_site_LCO.cfg for LCO location
Astrometry :
also get information on camera setup maybe it will be nesesary to flip frame for astrometry or put some specific rotation angle :

CCD_ASAS_ASTROMETRY_FI=-90.0

check pixscale for camera and change if need :

PIXSCALE=36.6

check if image flip is needed and change parameter :

CCD_ASAS_REVERSE_FOR_TRANSFORM=0

In case astrometry was already performed once and written to fits file there is no need to repeat it and the information from fits
header can be used in analysis , to do so

Check which location is choosen in ccd.cfg file , look of obs_site in ccd.cfg file , copy to obs_sits.cfg file on of these ( depending
on earth location ) :
        obs_site_brwinow.cfg
        obs_site_LCO.cfg

In order to save astrometry to all fits files during one run of ccddouble ( ccdsingle ) program set paramteres :
       CCD_SAVE_GOOD_ASTRO=1
       CCD_AUTO_ASTROMETRY_FREQ=1
NOTE : remeber to set CCD_SAVE_GOOD_ASTRO=0 for next analysis - otherwise you will write astrometry all the
time for all frames and it will take some time
After astrometry is written to fits files there is no need to do it on next analysis, in order to use astrometry from fits files set :
       CCD_USE_ASAS_ASTROMETRY_FROM_FITS=1
       CCD_DO_ASAS_PHOT_ASTR=0
fixed mount - settings for fixed mount in brwinow ( azimuth and altitude in radians ) :
1. CCD_OBS_MODE=0
2. CCD_HOR_AZIMUTH_IN_DEG=0
3. CCD_HOR_ALTITUDE_IN_DEG=38
4. in case stars are moving horizontaly or verticaly and there is no need to use rotation put CCD_DO_NOT_USE_ROT=1
satellites rejection :

In order to have satellites rejection workin you must make sure that you use most recent tle database file. Probably most
recent was produced when running night analysis and if you copied cfg files from template location then there is old tle file there
in order to have new one copy it from data location directory. Second thing to check is qth file check is satelitesdb.qth is good
for your location, otherwise overwrite it with one of template files satelitesdb.qth.LCO or satelitesdb.qth.WAW
sometimes the following situation can occure, that fits header coordinates was setup aproximatly with values for example
( AZIM , ALT )=( 0 , 38 )
but astrometry finds that in fact it was :
( AZIM , ALT )=( 1.23 , 39.3 )
in such situation program thinks that there was big change on coordinates ( in case CCD_USE_COORD_FROM_FITS=1 ) restars pipeline
and does not find flashes , in such situation best way is to setup coordinates in cfg file and turn off using of coordinates from fits file :
     CCD_USE_COORD_FROM_FITS=0
     CCD_HOR_ALTITUDE_IN_DEG=38
     CCD_HOR_AZIMUTH_IN_DEG=0
this will allow to analyse frames.

4. Analysis using FLT log files ( allevents_0.log and allevents_1.log ) :

In order to run analysis base only on First Level Trigger ( FLT ) log files first do changes described in point 3. and later at the following lines in the very begining of ccd.cfg file :

       CCD_READ_FIRST_LEVEL_INFO_FROM_LOG=1
       CCD_FROM_LOG_FILE_WITH_FRAMES=0
       CCD_FIRST_LEVEL_TRIGGER_DIR= path to directory where files allevents_0.log and allevents_1.log are stored
       CCD_USE_ASAS_ASTROMETRY_FROM_FITS=0

or simply include file offline_on_flt.cfg ( in $SRCDIR/ccd/cfg/WZORY_CCD_CFG/doubleCAM_Real/2K2K_2K2K )
Also remember to have astrometry files correct otherwise program claims that astrometry is missing it is not used just to have flag set that astrometry is OK - should be changed in future

5. Tn/Tv/T_shape

This 3 parameters are responsible for the following criteria :

PARAMETER NAME	DESCRIPTION	OLD SETTING	NEW SETTING
CCD_NEW_LAPLACE_TRESHOLD_IN_SIGMA	Tn - detection treshold for stars on new frame	4.0	8
CCD_MAX_LAPLACE_ON_OTHER_IN_SIGMA	Tv - veto treshold for stars on previuos frames	1.5	4/5
CCD_CLUSTER_IF_N_SIGMA_ABOVE_BACKGR	T_shape - treshold for finding cluster of pixels	1.8	5

After fit of gaussian always passes due to ROOT fit usage, treshold must have been changed. They were tested by LWP and settings which seem to be the best are shown in table above 4/5 means that it is 4 or 5. Complete list of results can be found here.
Suggested is setting Tn=8, Tv=4, T_shape=5 as more strict for background on bad nights, Tv=5 is also OK.

6. DAQ parameters.

This table summarizes daq parameters ( camera controll, flash recognition algorithms ) , all daq parameters are listed here.

7. DAQ parameter files

There are several pre-defined cfg files. Usage of files is described in chapter 1 , it is possible to use derectives LOAD and SAFELOAD
to load cfg file in another cfg file, SAFELOAD will continue in case file does not exist.
This table presents list of daq paremeters :

CFGFILE NAME	DESCRIPTION
ccd.cfg	main configuration file contatins : - loads specific cfg files - corba settings - algorithms settings - astrometry settings
device.cfg	device general settings
ccd_pipeline0.cfg ccd_pipeline1.cfg	overwrites settings in ccd.cfg specific for camera
device0.cfg device1.cfg	overwrites device settings for specific device
dynamic.cfg	this file is created dynamicaly when paremters are changed from piman, in case daq is restarted they are loaded from this file to continue with same settings as before restart of daq
/opt/pi/dev/pisys/daq/config/custom/ccd.cfg	machine custom settings can be set in this file , in case several locations have slightly differnet settings they can be put in this file
cam_and_lens.cfg	description of cameras optics etc, this is what is written to FITS header about camera
obs_site_LCO.cfg	observation site paremeters - geographical postition, altitude etc ...
single.cfg	settings for algorithm looking for flashes and requiring conformation on next frame
single_cam.cfg	same as single.cfg but should be used to run system on single camera in on-line mode ( analysing images directly from cameras ) , currently the only difference to single.cfg is that this file requires astrometry to be performed while single.cfg uses astrometry from FITS files

8. Handling non-typical situations :

in case we want to use only single camera for long period of time and run algorithm with flash confirmation on next frame , it can be best done
be copying content of file single.cfg into maching custom cfg file : /opt/pi/dev/pisys/daq/config/custom/ccd.cfg

9. Cooling strategy

Cameras are cooled in the very begining just after daq program ( ccdsingle or ccddouble ) is started, first thing it does after initialization is cooling of the cameras.
This is controlled by few parameters :

CCD_COOLING_ON_OFF - enables/disables cooling
CCD_TEMPERATURE - sets required temperature
CCD_WAIT_FOR_DESIRED_TEMP - specified time interval ( in seconds ) of waiting for desired temperature
CCD_COOLING_TOLERANCE_DIFF - tolerance value
CCD_ACTION_ON_TEMP_WAIT_FAIL - what should be done in case desired temperature was not reached , 0-set reached temperature , 1-continue cooling until desired temperature
will be reached

Defualt behaviour is that cameras are cooled to -10 degress in case after 1800 seconds they didn't reach this value and even not -8 degress ( abs(-10-(-8))<=2 ) it means
system sets reached temperature for example -5 deg and does not force to reach -10. In case -8 is reached it is in tolerance range of 2 degrees and camera continues cooling to -10.
It is possible to change this behavious by setting CCD_ACTION_ON_TEMP_WAIT_FAIL=0 which means that in case cooling failed camera will continue cooling to desired temperature
all the time during operation. Currently implemented actions are :
0-IGNORE ( and continue cooling , but stop waiting )
1-SET FINAL TEMP ( best reached temperature is set - for example -5 instead of -10 )
See also here for more details on paramters

10. ASAS Photometry/astrometry

Paramter file is PI.defaults cfg it is located in $SRCDIR/ccd/cfg in source directory tree and in $NDIR/cfg/ on target system