PI-Software

I/ Data AQuisition system

1. DAQ program

The main purposes of DAQ program is to collect frames from CCD cameras, on-line analysis to search for optical transients
Program is controlled by set of parameters defined in ccd.cfg file and ccd_pipelineX.cfg files. In case of current configuration in which
there are to twim cameras , camera specific parameters are stored in ccd_pipeline0.cfg and ccd_pipeline1.cfg
There is also possible to include other files in order to have different files for specific purposes like : device.cfg, obs_site.cfg etc ...
Program stors several previous frames ( CCD_PIPELINE_SIZE ) in order to compare new frame to combination ( in fact average ) of previous frames.
Frames are written to FITS files and by default compressed with ASAS-compression ( compr / ucompr programs ), information about FITS header is given here.

2. Flash recognition algorithms and parameters

Currently search for bursts uses several stages of new frame analysis:

Stage 1 ( very fast )
- apply Laplace filter to new frame and determine sigma, mean and calculate tresholds according to these values
- finding stars requires values on frame after Laplace to by greater then given treshold Tn, treshold is given in sigma. This cut finds ~40000 pixles belonging to stars on each frame.
- rejecting stars, this stage has to eliminate stars from further analysis in order to leave only stars which "just appered on the sky" and were not
  found on previous frames. For pixles which were found in step 1 average value L_prev - laplace on previous n frames is calculated and check against treshold Tv , only pixels with L_prev < Tv are accepted for further analysis. After this stage there is ~20 pixels left for further analysis
- Find clusters from remeind pixels and skip overlaping events
Stage 2 , cuts :
- black pixels
- hot pixels
- if more
- coicydence in 2 cameras
- tracks, there are several versions of this cut implemented, track can be found on single camera ( before coic ) , on single frame of single camera ( mainly for planes and meteors ) and after coic on many subsequent frames
- shape
- satellite database
- constant stars catalogue

This table summarizes daq parameter used by algorithms and off-line monte carlo simulation, all daq parameters are listed here.
More info about setting up astrometry, in order to fix astrometry in FITC headers see here
There are some details in this document also.

3. How to run and use

In order to run on-line analysis, do the following actions :

check if pikam driver module is loaded , login as root and do :
lsmod | grep pikam
in case driver is not loaded , load it ( still as root ) :
cd $NDIR/modules
insmod pikam.ko
prepCOIC_2K2K! will create new folder for today's data, get fresh satellite DB and copy currently used cfg files
cd $RESDIR/YYYYMMDD and run :
ccddouble
last 2 steps can be run as :
cd /opt/pi/dev/pisys
./run_daq DO_REDIR

There are several usful scripts and programs , this table summerizes them, convention is that scripts are named with ! or .sh at the end.

4. Running DAQ simulator

It is described here.

5. Running off-line flash recognition analysis

In order to run off-line analysis ( ccddouble or Nparamstest2C ) copy all *.cfg, sat* and *list* files to your prepared directory from YYYYMMDD/ANALYSIS/ directory
( for data older then 20040701 it is in fact YYYYMMDD/coic/ANALYSIS directory ). Then change parameters according to your needs and prepare mcinput.txt file if testing parameters will be performed with Nparamstest2C ( Nparamstest ) program
After data was put in differnet location then it was originaly the following changes may be required in order to run analysis :

ln -s /scratch/pi7/results/20040714/ANALYSIS/dark1.fitc
ln -s /scratch/pi7/results/20040714/ANALYSIS/dark2.fitc

edit ccd.cfg file and set :

CCD_FULL_FRAMES_SAMPLE_DIR=/scratch/pi7/results/20040714/

posibly change of dark frame file mayby required in ccd_pipelineX.cfg :

in ccd_pipeline0.cfg :
CCD_DARK_FRAME_FILE=dark1.fitc

in ccd_pipeline1.cfg:
CCD_DARK_FRAME_FILE=dark2.fitc

check by typing :

varhisto

Running monte carlo simulation in order to determine algorithms efficiency and bacground is described in detail here.

6. DAQ controll from pishell :

The following daq commands are currently available :

Command syntax	Command Type	General Purpose	Description
start_analysis( RA, DEC, AZIM, ALT, OBSMODE )	daq		Sets coordinates and starts analysis
stop_analysis	daq
set_temp( TEMP, CAM_IDX )	daq
get_temp( CAM_IDX )	daq
set_shutter_time( TIME_IN_SEC )	daq
get_shutter_time	daq
quit	daq
change_param( PARAM_NAME, PARAM_VALUE, CCD_INDEX )	daq
get_param( PARAM_NAME, CCD_INDEX )	daq
do_darks	daq
take_npictures( N )	daq
take_npictures_synchro( N )	daq		takes N picuters in synchro mode - waits for all to be finished before exit
set_cooling( ON_OFF , CCD_INDEX )	daq
get_cooling( CCD_INDEX )	daq
save_current_pictures	daq
stat	daq
get_position	daq
set_fits_key( KEYNAME, KEYVALUE, CCD_INDEX)	daq		In order to overwrite some FITS keys from pishell
start_daq	daq		In case parameter CCD_DAQ_STARTED was set to 0 on daq start , daq waits for this command to be called before proceeds with taking darks and frames
send_pos_to_daq	internal		Send position from mount to daq and start analysis
send_pos_to_mount	internal		Reads position from DAQ-astrometry and send to mount
send_pos_to_mount_astro	internal		Forces daq to make new astrometry and returns the result
update_cron	piman		updates corn information , it is listed here in case some daq commands are in cron and should be changed config/pi.cron can be edited and this command should be exeuted
clean_ast_list	daq	mount axis calibration	cleans list of ast files kept in memory, this list can be used to execute any kind of commands ( currently only mount axis calibration command ) NOTE : not yet implemented in piman/pishell
calc_mount_axis_calib	daq	mount axis calibration	according to collected ast files ( list kept in daq memory ) rotation axis of mount is calculated and shift from pole determined and returned to user, also screw moves in mm are determined according to information stored here ( from first manual mount calibration ) , pixscale is used from daq ccd.cfg file so it is taken into account that new objectives can have different pixscale Special script was developed to be run under piman NOTE : not yet implemented in piman/pishell
send_pos_to_daq_if_astro_failed	internal		sends position to DAQ is astrometry failed and starts analysis
fast_pos_to_daq_if_astro_failed	internal		sends position to DAQ is astrometry failed - no start analysis , only position is sent
point_best_target	piman		finds best target to be followed, position change is not executed if distance from previous position is <10 deg
forced_point_best_target	piman		finds best target to be followed, position change is always executed , also when distance from previous position is <10 deg

Where :
OBSMODE = 0 - mount const positon ( AZIM , ALT )
1 - tracking mode

CCD_INDEX = 0/1 - selecting specific cameras
-1 - all cameras

ON_OFF = 0/1

Usage example :
To change Tn treshold run :
daq 0 change_param(CCD_MAX_LAPLACE_ON_OTHER_IN_SIGMA,5,-1)
change Tv treshold :
daq 0 change_param(CCD_MAX_LAPLACE_ON_OTHER_IN_SIGMA,2,-1)
changing shutter time to 5 sec :
daq 0 set_shutter_time(5)

NOTE : It is also possible to call daq functions without usage of piman and pishell , but using sendreq2 program which was
created for daq-corba communication testing purposes, in case some function is missing in piman it can be called using
this program

7. Setting up DAQ program and actions on problems

This is described in detail here.

8. Setting up DAQ parameters

Location of daq configuration files is described in this document, cfg files are described here
Setting of daq parameters is described here.

9. Building daq:

login as root execute :
   mkdir /opt/pi
   chown pi /opt/pi
   chgrp pi /opt/pi
login as pi , execute :
mkdir -p /opt/pi/dev/pisys/daq/
go to directory /opt/pi/dev/pisys/daq/
checkout daq from subversion repository :
svn checkout svn://heplx47/opt/pi/svn/pi/trunk/daq .
Przekopiowac z innego kompa z ta sama wersja systemu katalog /opt/pi/ext/ :
cd /opt/pi/
scp -r pi@SOURCE_COMP:/opt/pi/ext .
edit file ~/.bashrc and add line :
. /opt/pi/dev/pisys/daq/src/env/ccd_env_opt
In fact it should be present in file /etc/profile ( but in case it is not there add to .bashrc )
re-login
compile daq :
cd $SRCDIR
make clean
make all
Migration to gcc4.0 is described in this document.
Configuring log4c is done in :
/etc/log4crc
and here syslog.conf example

10. Taking flats

Is described here

11. Current results directories structure

Is described here

12. Pi programs and scripts

Are summerized in this table.

13. Structure of pi software ( also subversion structure )

Structure of pi software under subversion is described in this document.

14. Running daq manually

a/ Single-Camera USB mode

check USB driver module is already loaded :
su -
lsmod | grep pikam
in case it is loaded skip next command and continue with starting daq, otherwise turn on camera and
execute the following step ( still as root ) :
loadpikam!
then check if it was loaded correctly ( as above : lsmod | grep pikam )
login as pi in another console, rest of actions should be performed as pi used
prepare catalog for collecting data, configuration files and retrive satellite information from WWW :
   - login as pi user and execute script :
       prepdaq_single.sh
   In case running in Brwinow use : prepdaq_single.sh BRW
    this may take some time due to fact that tle files are retrived from web
On order to use cfg files for camera other then Carl Zeiss 50 mm use 2-nd parameter, for example to use Cannon 85 mm :
      prepdaq_single.sh BRW cannon_f85mm
Always usage can be printed by :
      prepdaq_single.sh -h
check if everything was created :
   cd $RESDIR/
   cd YYYYMMDD
   ls *.cfg
   ls -al *.tle
check if camera is working fine , without camera settings initialization :
     test2K2K k2a
or with camera settings initialization :
   test2K2K k2a -init_params
in case you need more options use ( always you can check options using : test2K2K -h )
     test2K2K k2a -init_params -medium
in order to take dark frames in test2K2K , put negative shutter time
run frames collection with flash recognition algorithm using confirmation on next frame :
cd $RESDIR/YYYYMMDD/
corba_nsd!
nohup runccdsingle! > out 2>&1 &

b/ Double-Camera USB mode

check USB driver module is already loaded :
su -
lsmod | grep pikam
in case it is loaded skip next command and continue with starting daq, otherwise turn on camera and
execute the following step ( still as root ) :
loadpikam!
then check if it was loaded correctly ( as above : lsmod | grep pikam )
login as pi in another console, rest of actions should be performed as pi used
prepare catalog for collecting data, configuration files and retrive satellite information from WWW :
   - login as pi user and execute script :
       prepdaq.sh
   In case running in Brwinow use : prepdaq.sh BRW
    this may take some time due to fact that tle files are retrived from web
   On order to use cfg files for camera other then Carl Zeiss 50 mm use 2-nd parameter, for example to use Cannon 85 mm :
      prepdaq.sh BRW cannon_f85mm
Always usage can be printed by :
      prepdaq.sh -h
check if everything was created :
   cd $RESDIR/
   cd YYYYMMDD
   ls *.cfg
   ls -al *.tle
check if camera is working fine , without camera settings initialization :
     test2K2K k2a
or with camera settings initialization :
   test2K2K k2a -init_params
in case you need more options use ( always you can check options using : test2K2K -h )
     test2K2K k2a -init_params -medium
in order to take dark frames in test2K2K , put negative shutter time
run frames collection with flash recognition algorithm using coincydence on both cameras :
cd $RESDIR/YYYYMMDD/
corba_nsd!
nohup runccddouble! > out 2>&1 &

c/ Testing in Brwinow

Before taking heplx42 to Brwinow the following actions should be performed when heplx42 is still in hoza network :

check which disks are really needed to avoid demage in transport ...
turn off network services to avoid problems when heplx42 is not in hoza network , login as root , choose "System Settings/Server Settings/Services"
from "Start" ( red hat buton on toolbar in Fedora Core 2 - gnome ) , then uncheck the following services :
       nfs
       nfslock
       autofs
       sendmail
       netfs
       cups
do it in Runlevel 5 then do "Save" choose "Edit Runlevel"/Runlevel 3 , uncheck same services and save, this is very important, otherwise
everything works very slowly, booting is problematic
in order to overwrite some daq parameters put specific settings in ccd.cfg or device.cfg files in directory :
/opt/pi/dev/pisys/daq/config/custom/
values placed in this files will overwrite normal settings, check For for more details

15. Camera driver documentation

     Document describing camera commands can be found here ( old version ) . There exist common API library for using cameras this is libccddriver.a
     library developed by Marcin Sokolowski , any type of camera application uses this library. However in case of different types of cameras
      different lower level driver is used, in case of USB2.0 camera kernel module pikam.ko is used ( developed by Michal Jegier ) and
      in case of ethernet gigabit camera CEthCamera library ( using C++ Sockets Library ) is used.
        Quick introduction to building of standalone driver version is here and about using program test2K2K ( in USB and ethernet mode ) can be found in README.txt and USAGE.txt.

     Currently there are several versions of kernel module pikam.c :

     drv2.6.5/ - version for kernel 2.6.6 and 2.6.5
     drv2.6.6new/2.6.18 - version for kernel 2.6.18 - after compilation corrections
     drv2.6.6new/2.6.22 - version for kernel 2.6.22 - some compilation corrections ( WARNINGS )

    On FC7 version from drv2.6.6new/2.6.22 must be used, troche moich zapisow z tych zmian :
            heplx46 - sprawdzic kamere pod usb !
   Na razie sie przekompilowalo pod FC7 - 2.6.22 i sie laduje ,ale byl jakis
warning , pytanie czy tak moze zostac ???
Trzeba to zbadac - bo moze sie cos kisic
     /home/8/msok/WWW/kernel/fc7.txt
   arg liczone od 1 chodzi wiec o ten : skel_write_bulk_callback

wyglada na to ze wystarczy dodac casta do usb_complete_t , nie wysatarczy
bo w nowym jadrze taka jest definicja :
     typedef void (*usb_complete_t)(struct urb *);
czyli wystarczy zmienic definicje tej funkcji i bedzie OK !
Wyciac te regsy i bedzie OK

   TESTY na FC7 wykonalem na heplx46:/opt/pi/dev/pisys/daq/ndir/data/TEST/ obrazki sa sensowne, wszystko dziala poprawnie, sciagaja sie itd, itd

1. USB2.0 camera driver

2. Gigabit ethernet camera driver

     Library for communication with ethernet camera is the same as one for communication with USB2.0 camera ( libccddriver.a - developed by Marcin
     Sokolowski ) , however there is new lower level library for communication with ethernet camera through NUDP protocol. This library was developed by
    Robert Sulej documentation can be found here ( mirror )

16. DAQ status file

DAQ program dumps status file after every image is collected, by default it is saved to file /opt/pi/dev/pisys/status/daq.status , the meaning is the following :

	[GENERAL]
	DTM        = 20061014_034802            - DATE TIME ( LOCAL )
	UNIX_TIME  = 1160812082                 - UNIX TIME
	START_DTM  = 20061014_021606            - START DATE TIME (LOCAL )
	WAITING    = 0                          - IF IN WAITING MODE
	DOME       = OPENED                     - DOME STATUS
	FAST_MODE  = 1                          - IF FAST ( ASYNCHRO ) COLLECTION MODE , IMAGE IS TRANSFERED WHEN NEXT IS BEING TAKEN
	MODE       = NORMAL                     - OPERATION MODE
	ALERT#     = 0                          - NUMBER OF ALERTS
	AUTOGUIDE  = 0                          - IF AUTOGUIDING MAY BE ENABLED ( ASTROMETRY OK )
	PROGRAM    = ccdsingle                  - NAME OF PROGRAM ccdsingle or ccddouble

	[CAMERA_k2a]
	FRAME#     = 262                        - NUMBER OF FRAMES COLLECTED
	ASTROMETRY = 0                          - LAST ASTROMETRY RESULT
	STARS#     = 322                        - NUMBER OF STARS DETECTED ON IMAGE
	FWHM_AVER  = 0.00                       - AVERAGE FWHM ON FRAME ( IN FAST PHOTOMETRY MODE NOT YET CALCULATED )
	RA         = 292.59                     - RA [ DEGREES ]
	DEC        = -40.00                     - DEC [ DEGREES ]
	AZIM       = 32.46                      - AZIMUTH [DEGREES ]
	ALT        = -9.98                      - ALTITUDE [ DEGREES ]
	MODE       = EARTH_ROT                  - MODE OF MOUNT ( TRACKING OR CONST ) ASSUMED BY DAQ - USED IN ALGORITHMS ( IF CALCULATE SHIFT OR NOT )
	DEVICE     = OK                         - DEVICE STATUS
	CHIPTEMP   = -10.00                     - CURRENT CAMERA TEMPERATURE
	CHIPTSET   = -10.00                     - EXPECTED TEMPERATURE
	COOLING    = OK                         - IF COOLING OF CAMERA SUCCEDED WHEN DAQ WAS STARTED
	ERRCOUNT   = 0                          - NUMBER OF ERRORS
	REPEATED   = 0                          - NUMBER OF REPEATED IMAGES
	LAST_ERROR =                            - LAST ERROR
	FITC_SIZE_Kb= 4353.08                   - SIZE OF FITC FILE
	PREV_ASTRO  = 0                         - RESULT OF PREVIOUS ASTROMETRY
	FAILED_ASTRO= 0                         - NUMBER OF SUBSEQUENT FAILED ASTROMETRIES

	[MOUNT]                                - MOUNT POSITION - AS DAQ THINKS IT IS 
	MOUNTRA    = 292.59                    - RA
	MOUNTDEC   = -40.00                    - DEC
	MOUNTAZIM  = 38.98                     - AZIM
	MOUNTALT   = -4.18                     - ALT
	MOUNTTRK   = 1                         - TRACKING YES/NO ( 1/0 )
	MOUNTTIME  = 1160809526                - UNIX TIME TIMESTAMP OF THIS DATA

17. Simulation - puting samples on sky images

This is mainly to test efficiency of flash recognition agorithms. Samples of stars of given magnitudo are pasted on image. Sample information is saved to log file genevents_0.log
Coordinates of maximum pixel are saved there, in case of testing confirmation of next algorithm same sample is then re-pasted on next image.
I was trying to put another random sample , but it was not good - mayne it can be changed but now it is better to put same sample ( different maximum position on other sample
changes the results of confirmation on next check ). Without puting same sample a lot of generated events were rejected on confirmation on next step , which was due to fact that
different sample was put which had maximum shifted and confirmation rejected such sample !!!
It is possible to save image with pasted samples by setting paramter :

                CCD_SAVE_IMAGE_WITH_SAMPLE=1

Running simulation on single camera with confirmation on next algorithm :

             nohup ccdsingle -putsample=9.00 >putsample.out 2>&1

it can be used instead of Nparamstest but paramters must be changed by putting $(new_value) in ccd.cfg file and changing it in loop in script
Then in RESULTS/20070101_100000 pieffbkg program checks finalevents_0.log and genevents_0.log and finds Eff and Bkg !
Some recent improvements of simulation are described here.
Script for running tests of parameters is run_eff_bkg_test! .

II/ PI-system

1. Starting system :

System is started by run_pisys! script , it take parameters :

run_pisys! dome_status 40 noinit

dome_status - checking dome status before starting , only if dome is opened , continue
40 - default number of checks performed every 15 minutes
noinit - no pish script is executed, only servers are started

Starting DAQ has several steps :
- directory YYYYMMDD is created in /data2/results/, and cfg files are copied from /opt/pi/dev/pisys/daq/src/ccd/cfg/WZORY_CCD_CFG/doubleCAM_Real/2K2K_2K2K
this means that changes eneterd in ccd.cfg file in that location will take place next night
- satellites datebase is retrived from known sources
- runccddouble! script is started, it restarts ccddouble program after 60 sec in case of unexpected crash ( but max 10 times during a night )
- daq remains in waiting state , until piman sends message that mount is on position , after this moment analysis is started

2. Crontab :

Current crontab settings are described here

3. System reporting options

Every day at 9 AM pi-system morning report is generated, and e-mail is send to people which wanted to obtain this kind of report.

4. Boot On LAN

Is described here

5. Init.d/pi - pi services launched on system statup

There are several things that must be started when pi2 computer is started , like loading camers driver ( pikam.ko ) , loading firmware to k2b camera , all these are described here.

6. Drive configuration and changing of rack drives at LCO

Is described here

7. Computers and systems @hoza

See here.

8. Troubleshooting

Is described here.

9. GCN log files

10. Mount axis calibration procedure

The purpose of mount axis calibration procedure is to determine shift of mount rotation axis from earth rotation axis, we would like
to have it exactly parallel, however it is not easy to do it perfectly.
The procedure goes to field near the pole ( on south pole 0000-75 was choosen , however any other field can be used ), it is not
working when pointing directly to the pole due to problems with astrometry at this position which could not be resolved yet.
After moving mount to this position single image is taken and astrometry is performed, then mount rotates by desired number of degrees ( currently 30 deg )
in RA axis and next photo/astrometry is taken, then next rotation by 30 deg and next photo/astrometry.
When we have 3 files with astrometry common stars on all images are found and rotation center is determined as average position of
rotation center of all common stars.
The following pish script performs the above procedure.
Fields :
0000-75 - good , tested frame was rotated using command : calcfits k2a_041201_000.fitc R k2a_041201_rot60.fitc -rotate=1010,1900,60
                 First rotation around point (1028.26,1881.47) near the south pole (1027.88,1913.35) was tested and rotation 30,60 degrees was done using
                 command above, center of rotation was determined at (1027.77,1880.75) so the agreement is quite good.
                 Rotation around pole itself was also tested and gave good results too.
0712-75 - not good - south pole is outside this field
0224-75 - good, tested with rotation around (1010,1900) 30 deg and 40 deg , rotation center calculated was (1010.56,1900.64), south pole was
                at (1027.25,1945.70), calculated shift was (16.68,45.07) pixles which means (2.09,5.63) mm of screw move

Example of same procedure without piman usage ( only daq commands ) , it was tested on heplx43 with test frames near of pole taken
from scan , and rotated using option of program calcfits2 in heplx43:$RESDIR/simul/ccddouble/mount_calib/

11. Cloning linux system ( especially pi1 and pi2 )

For procedure for LCO trip see here ( pdf ).
Nice description can be found here , here are sumerize steps :

create partitions on new hard disk ( I assume it is /dev/sda ), remember to set bootable partition flag
on first partition where system will be installed ( it can be done in fdisk using command 'a' )
run konppix system and copy old system root directory ( / ) to new disk :
     su -
    mkdir -p /mnt/old_system/
    mkdir -p /mnt/new_system
     mount /dev/sda1 /mnt/new_system -t ext3
     cp -a /mnt/old_system/* /mnt/new_system/
install grub in MBR of new disk ( more info on grub-install here and here , or for manual grub instalation see here ) :
     grub-install /dev/sda1 --recheck --root-directory=/mnt/sda1/
           or
     grub-install /dev/sdX --recheck --root-directory=/mnt/sdaX/

PI1 :

new sata disk 80GB was partitioned in WAW :
...
now we must copy current pi1 system partition to the first ( /dev/hda1/ 20GB ) partition, run system from knoppix cd , open terminal,
loging as root ( type : su - ) , mount old and new system :
    mkdir /mnt/old_system
    mkdir /mnt/new_system
    mount /dev/hdc1 /mnt/old_system
    mount /dev/sda1 /mnt/new_system
    cp -a /mnt/old_system/* /mnt/new_system

12. Creating RAID and Logical Volumes

RAID can be based on hardware controller which is good solution as disk is visible as single big drive, hovewer this is quite expensive
solution as RAID controller cost starts at 600 PLN and goes up to 2000 PLN.
Creation of Software-RAID and Logical Volume is described here.

13. Usfull tips and asas commands for LCO system

operating asas dome
In order to changing asas dome operation to manual mode , to avoid automatic closing of dome when manipulating with cameras login
on asas3m ( or asas3v ) machine and type :
db_set obsrv_status SLEEP -dbserver asas3v
in order to change back to automatic mode type :
db_set obsrv_status RUN -dbserver asas3v
in case db_set program is not available from the command line check here : /home/asas3m/data/gp/bin/db_set

14. Log files location and viewing

General rule is that smaller jobs output is written to /opt/pi/dev/pisys/log/crontab.trace and bigger jobs have own log files ,

TABLE1 : pi2 log files

Program Name	Log File Location	Description
MAIN PI SYSYTEM LOG	/opt/pi/dev/pisys/log/pi.log	In order to turn off loging to main system error in daq edit file : /opt/pi/dev/pisys/daq/config/custom/ccd.cfg on pi2 and set : CCD_GLOBAL_LOG_ENABLED=0
killdaq.sh	/opt/pi/dev/pisys/log/crontab.trace	kills daq processes in the moring in case hanged and didn't exit on piman request
cooling_off.sh	/opt/pi/dev/pisys/log/crontab.trace	disables cooling in case daq failed to do it on exit
run_piphoto!	/opt/pi/dev/pisys/log/piphoto.trace	starts fast photometry
prepdaq.sh	/opt/pi/dev/pisys/log/prepdaq.log	prepares directory for new night data, retrives satellites databases from www, copies cfg files to night directory
autogen_cron.sh	/opt/pi/dev/pisys/log/genscript.log	generates night script YYYYMMDD.pish , calls genscript program
run_pisys!	/opt/pi/dev/pisys/log/pisys_start.log	starts pi system in the evening
runslt!	/opt/pi/dev/pisys/log/slt.log	starts SLT ( Second Level Trigger )
cplogs!	/opt/pi/dev/pisys/log/crontab.trace	copies night log files to night data directory to have them in one place with data
cat_all_night_events.sh	/opt/pi/dev/pisys/log/crontab.trace	cats final events from night to single log file
clean_online.sh	/opt/pi/dev/pisys/log/crontab.trace	cleans data on data collecting machine , night fitc data is removed after couple of days
prep_morning_report.sh	/opt/pi/dev/pisys/log/crontab.trace	prepares morning report and sends e-mail
start_monitor.sh	/opt/pi/dev/pisys/log/crontab.trace	starts jobs which create jpg files , takes status files and packs all stuff to tar.gz file which then is copied from heplx40 and placed on www server This script runs several scripts which have own log files listed below , up to stop_monitor.sh line :
last2jpg!	/opt/pi/dev/pisys/log/frame_monitor.log	converts last collected frame to jpg file
update_alert_html.sh	/opt/pi/dev/pisys/log/html.log	converts events to jpg file
prep_status_pack!	/opt/pi/dev/pisys/log/status_monitor.log	prepares pack of status files ( tar.gz file ) to be transfered to hoza
monitor_memory!	/opt/pi/dev/pisys/log/memory.log	monitor of system memory, not very usfull
stop_monitor.sh	/opt/pi/dev/pisys/log/crontab.trace	executed in the morning, stops processes started with start_monitor.sh script
prepare_to_take!	/opt/pi/dev/pisys/log/crontab.trace	????
totxt.sh	/opt/pi/dev/pisys/log/crontab.trace	dumps fits headers to txt files
send_asas_info_mail.sh	/opt/pi/dev/pisys/log/crontab.trace	sends e-mail about asas pipeline statistics
get_super_novea.sh	/opt/pi/dev/pisys/log/get_sn.log	gets SN information to database
update_gcn.sh	/opt/pi/dev/pisys/log/update_gcn.log	updates database of grb information from www
pidb_dump_new_night.sh	/opt/pi/dev/pisys/log/pidb_dump.log	dumps new night data from db to sql file to be loaded from remote to synchronize database
checkext.sh	/opt/pi/dev/pisys/log/checkext.log	checks coincydence of pi alerts with external alerts - not very usfull now
archlog.sh	/opt/pi/dev/pisys/log/arch.log	archivize log files
run_offline.sh	/opt/pi/dev/pisys/log/offline_start.log	start offline analysis
run_gcn_server.sh	/opt/pi/dev/pisys/log/crontab.trace	restarts gcn server , it is done every afternoon
clean_online_monthly!	/opt/pi/dev/pisys/log/clean_monthly.log	cleans temporary converter files and bacups log files from previous month
do_object!	/opt/pi/dev/pisys/log/do_object.log /data2/results/20061022/test_param_20061022.log	Checker of keyword OBJECT in fits headers
deep_purple	/data1/results/log/deep_purple	log files from deep purple criteria check ( also same location on pi3 ) File with names deep_purple_16_20070420.out where 16 - means io_type and 20070420 - night

Table 2 : pi1 log files pi-pipeline daily run - cataloging of 20 averaged images

Program Name	Log File Location	Description
start_monitor_pi1.sh	/opt/pi/dev/pisys/log/crontab.trace	restarts monitoring jobs : run_request_handler!
run_asas_pipeline_online_cat! Options in crontab : run_asas_pipeline_online_cat! - /data2/results /data1/results/PI_PIPELINE/ aver20_2006 - pi1.lco.cl	/opt/pi/dev/pisys/pipipeline.log	pi-pipeline daily run - cataloging of 20 averaged images it is run daily more information can be found in log files specific for night in : pi1:/data1/results/PI_PIPELINE/YYYYMMDD : asas_pipeline.out - log of reduction/photometry and astrometry cataloging.out - log of cataloging
run_request_handler!	/opt/pi/dev/pisys/log/request_handler.log	Request handler on pi1 , started by script start_monitor_pi1.sh , restarted every day at 19:45 Log kazdego z requestow znajduje sie tutaj : /opt/pi/dev/pisys/requests/ w odpowiednim podkatalogu : GETPART_19.028867_1.437_20060605 i w pliku : GETPART_19.028867_1.437_20060605.log Pozniej wyciniki sa robione w katalogu z noca w podkatalogu : Event_..... tam tez jest plik loga getpart2.log Natomiast na heplx40 wyniki sa zapisywane w : ~/www/pi0/daq/requsts/20060606 ~/www/pi0/daq/events/grb/grb20060606 i te katalogin nalezy skasowac jesli chcemy ponowic transmisje
update_scan!	/opt/pi/dev/pisys/log/update_scan.log	synchronizes scan fits files after astrometry is performed in the morning on pi2 and saved to fits files headers
find_flares.sh	/opt/pi/dev/pisys/find_flares.log	output from flare finder algorithm
compress_final!	/opt/pi/dev/pisys/log/compress_final.log	output from script compressing final events parts , files of type : Frame01234.tar.gz are created in RESULTS/2*/Events/ subdirectory

Table 3 - pi3 log files :

Program Name	Log File Location	Description
Fast photometry log file on pi3	/opt/pi/dev/pisys/log/fast.log

Table 4 - heplx40 log files :

Program Name	Log File Location	Description
get_status_html!	/opt/pi/dev/pisys/log/status_update.log	getting status files from pi2
get_alerts_html!	/opt/pi/dev/pisys/log/html_update.log	getting alerts jpg from pi1/pi2
get_frames_html!	/opt/pi/dev/pisys/log/html_frames.log	getting frames jpg from pi2
synchro_db.sh	/opt/pi/dev/pisys/pidb/pi2/synchro_db.out	synchronizing DB from pi2

Log files from mount , piman and daq are possible to view using scripts : mounttrace! , pimantrace! and daqtrace! .
Every night these log files have different names containing night date.

15. UPS usage

UPS usage and monitoring is described in seperate document.

16. System in single camera mode

In case of problems with one of the cameras it is still possible to run system using one camera. The procedure how to do it is written in this document.

17. Backups at LCO

Backuped data	Backup Location	Description
system log files	pi2:/opt/pi/dev/pisys/log/backup/	log files are backuped on first day of new month, subdirectory YYYYMM is created in the backup directory and files are copied there and compressed

18. Loading firmware and vhdl to camera

It is possible to upgrade camera firmware using script load_cypress.sh ( or load_cypress! , see here for more details ) and it is also possible to load vhdl to altera - using script load_altera!. However loading of vhdl is not currently working, it is waiting to be solved by GK. I tried to do it, but I didn't suceeded
, it is described in this document.

19. Manipulation of dome status on pi computers

Manipulation of dome status is described here.

20. Building pi system

building mount :
   cd ./mount
   ln -sf makefile.corba makefile
   make clean
   make
   #
   # copy all essential config files
   #
   cp -f bin/config/mount.config     ~/opt_pisys/config/.
   cp -f bin/config/mount.calib_last ~/opt_pisys/config/.
   cp -f bin/config/mount.locations ~/opt_pisys/config/.

   ...a potem trzeba tylko odpalic PIMANa i
   zrobic 'calib 0' albo 'calib 1'.

21. Getting parts of images

Images from several recent nights are stored in pi1 computer, it is possible to get parts of images at given position by sending requst from this page.
NOTE : be patient !!!!!!! Sometimes astrometry must be done on series of images
Getting of parts is done by "server" script running on pi1 it is run_request_handler! script is is restarted every day by crontab by script start_monitor_pi1.sh , crontab line :

45 19 * * * /opt/pi/dev/pisys/daq/ndir/bin/start_monitor_pi1.sh >> /opt/pi/dev/pisys/log/crontab.trace 2>&1

in order to check log file from this server run command handler! on pi1 ( it prints also name of file - so it can be opened in joe, less etc ... ) . This works on basis of file copying by scp, after submiting request by web page script exec_request! is executed on heplx40 as user piwww. Results are stored in directory ~/www/pi0/daq/events/ and subdirectory according to event type ( grb/nova/other/test ) . In case something goes wrong :

check run_request_handler! log file on pi1 , check specific request log file ( look for signs of last request in /opt/pi/dev/pisys/log/request_handler.log )
run "tail -f" on this request specific log file
in case request was already done, check if it was finished on heplx40 :
ps -ef | grep exec_request
check if results were retrived from pi1 in ~/www/pi0/daq/events/ if not you can donwload them manualy , or try again from web page, if it is saying that
request was already executed , go to ~/www/pi0/daq/events/REQUEST_TYPE/ and remove subdirectory named as REQUEST_NAME ( for example GRB061019 )
sometimes restart of request handler on pi1 is needed run :
/opt/pi/dev/pisys/daq/ndir/bin/start_monitor_pi1.sh
if program says :
another request is currently running on pi1
login on pi1 , verify handler! log file, if nothing is being processed now, then remove request file :
rm -f /opt/pi/dev/pisys/requests/request.lst

ONCE MORE : be patient !!!!!!!!!!!!!

22. Pointing of telescope

Pointing is now performed dynamically, there were recently ( 200704 ) added new paramters which may be set in $NDIR/cfg/ccd.cfg on pi2 in order to set pointing target priority and special pointing target. Pointing to satellites uses log files from gcn_server program which are :

SWIFT : swift_pointdir.log
INTEGRAL : integral_pointdir.log

In order to modify pointing paramters dynamically from pishell ( or runscript ) see table below for piman commands :

Pointing parameter name	Description	Default value	Piman Command
CCD_AUTOPOINT_SWIFT	enable/disable SWIFT pointing	1	-
CCD_AUTOPOINT_INTEGRAL	enable/disable INTEGRAL pointing	1	-
CCD_POINT_IO	enable/disable InterestingObjects pointing	1	-
CCD_POINTING_PRIOR	set priority of satellites	SWIFT,INTEGRAL	set_pointing_prior( INTEGRAL,SWIFT )
CCD_SPECIAL_TARGET	set special target ( with highest priority )		set_special_target( 12.32, -20.00, SN2006mq )

III/ Operating cameras

1. Focusing procedure

In order to perform focusing to the following actions , goto directory TEST , example :

1/ take picture with k2a, move k2a by -10 steps and take new picuture :
cd /data2/results/20040901/TEST/
k2a_sky!
getpart2 k2a_040901_00001.fitc - -part_size=100 -x=1000 -y=1000
movefocus -id=2 -steps=-10
k2a_sky!

2/ same for k2b :
cd /data2/results/20040901/TEST/
k2b_sky!
getpart2 k2b_040901_00001.fitc - -part_size=100 -x=1000 -y=1000
movefocus -id=3 -steps=-10
k2b_sky!

!!!!!!!!!!!!!!!
WARNING : script k2a_sky! / k2b_sky! is not protected when cameras , in such case instead of k2a_sky! use k2b_sky! and opposite
NOTE : movefocus is proof for this - it selects cameras by ID, so 2 ALWAYS means k2a and 3 is k2b !
!!!!!!!!!!!!!!!

2. Loading firmware to camera ( cypress soft )

3. Running real analysis and gigabit ethernet driver

Running real DAQ analysis is described here in details. Gigabit ethernet driver and its usage is described here.

IV/ Off-line analysis

1. Analysing choosen objects

In order to get parts of several frames in specified position one has todo the following action :

    a/ maunal way
    - choose frame from which object will be cut off
    - run the following programs :

     piad2xy frame.fit 12.34 -2.34 -from_fits

   this will give X,Y coordinates of desired object
       - prepare list of frame from which object will be taken from ( object_list_1 )
- run command :
            getpart2 object_list_1 - -part_size=100 -dark=dark1.fitc -x=X -y=X -nosubtr -prefix=k2a_-outdir=OBJECT
- in case also sum of this frames is required for further analysis do:
            cd OBJECT
            calcfits2 -list=list -oper=0 -out=sum_all.fitc

    b/ automatic way
    - go to directory where frames are stored and prepare a list of frames from which object will by cut off ( object_list_ccd1, object_list_ccd2 )
- run command :
            getpart2 object_list_1 - -part_size=100 -dark=dark1.fit -nosubtr -prefix=k2a_ -outdir=OBJECT1/k2a/ -ra=0.419722 -dec=12.287
            getpart2 object_list_2- -part_size=100 -dark=dark2.fit -nosubtr -prefix=k2b_ -outdir=OBJECT1/k2b/ -ra=0.419722 -dec=12.287

    in case sum of all parts is needed :

            getpart2 object_list_1 - -part_size=100 -dark=dark1.fit -nosubtr -prefix=k2a_ -outdir=OBJECT1/k2a/ -ra=0.419722 -dec=12.287 -sum
getpart2 object_list_2 - -part_size=100 -dark=dark2.fit -nosubtr -prefix=k2b_ -outdir=OBJECT1/k2b/ -ra=0.419722 -dec=12.287 -sum

NOTE :

ra - in hours, dec in degrees

In case of problems with astrometry definition ( no ccd_astrometry_pipeline0.cfg and parametrs are not defined in fits file ) , it will be nessesary
to run astrometry to do it easily script astro_k2a! or astro_k2b! can be used in the following way :

        astro_k2a! k2a_040923_02000.fitc
astro_k2b! k2b_040923_02000.fitc

after this command file ccd_astrometry_pipeline0.cfg will be created and points a/ and b/ will work fine

2. Fast photometry

Every day at 8 AM fast photometry pipeline is started on pi2.lco.cl machine, script run_piphoto! is executed, in case parametrs of photometry should be changed, like for example treshold for stars
script runPhotometry! should be edited :

    cd $SRCDIR/ccd/scripts/shell
joe runPhotometry!

correct value of treshold should be entered in the lines :

piphoto - -dark=$DARK1 -dir=$OUT_DIR -flat=$FLAT_K2A -list=full_frames_list_ccd1 "$GZIP_OPT" -flip -tresh=3.5
piphoto - -dark=$DARK1 -dir=$OUT_DIR -flat=$FLAT_K2A -list=full_frames_list_ccd1 "$GZIP_OPT" -flip -tresh=3.5

currently treshold = 3.5 sigma is used, after file is edited and saved type :

make all

Change will be used when auto-running from cron next time

Running :
    Script should be executed in directory where frames and dark is stored, by default $DATADIR/FLAT/FLAT_k2a and $DATADIR/FLAT/FLAT_k2b are used, script takes the following parameters :

          runPhotometry! RUN_DIR OUT_DIR GZIP_OPT SUB_DIR FLAT_k2a FLAT_k2b

   In case no paremetrs are provided RUN_DIR equals to current working directory (CWD), OUT_DIR=SUB_DIR in CWD , by default GZIP=1 and flats as above
   Minuses should be used to leave default value unchanged
   Photometry also flips images according to value of parameter CCD_ASAS_REVERSE_FOR_TRANSFORM
   To check results see here

More details on photometric algorithm are given here.

3. Creating ASAS-star catalogs ( OBSOLATE )

In order to create catalogs do the following steps :

create astrometry files ast , by running script :

pi_photo_astro! AST_LIST_FILE TRESHOLD

AST_LIST_FILE - name of file containng list of ast files to be catloged
TRESHOLD - treshold for photometry

This will run PI-photometry and then astrometry, in case one wants to use ASAS-photometry script asas_photo_astro! , usage is the same as for pi_photo_astro!
after ast files are created they should be gropued in the following directory structure :

ast/k2a - ast files for k2a
ast/k2b - ast files for k2b
first of all lists of ast files should be created in for k2a and k2b :

cd ast/k2a
ls k2a*.ast > k2a_ast_list
cd ../../ast/k2b/
ls k2b*.ast > k2b_ast_list

and from directory containing ast subdirectory the following script should be executed :

run_cat_fast_all.sh k2a_ast_list k2a 50
run_cat_fast_all.sh k2b_ast_list k2b 50

50 - means minimum number of measurements required for light curve to be created
4th - paramter is possible to indicate SEED_FRAME , instead of choosing max-size ast file

4. Running fast photometry and cataloging

In order to run fast photometry pipeline and cataloging use scripts :

red_cat_fast_photo!	Parameters : NIGHT - 20050420 OUT_DIR - out directory ( without NIGHT ) START OLD - 1 in order to process old nights also DATA DIR - data directory ( without NIGHT ) DATEBASE - database name	runs fast photometry/ astrometry and pi-cataloging
run_piphoto!	Parameters : NIGHT - 20050420 OUT_DIR - out directory ( without NIGHT ) START OLD - 1 in order to process old nights also DATA DIR - data directory ( without NIGHT ) DO_GZIP - if gzip mag/ast	Runs photometry and astrometry and places results in proper directories. Gzips results on request ( 5-th parameter )
cat_fast_photo!	PARAMETERS : 1 - FAST PHOTO DIR ( without night ) 2 - NIGHT - 20050420 3 - database name	Runs cataloging of ast files in given directory to specified database.
cat_nights_scan_new!	PARAMETERS : 1 - directory with ast scans 2 - options [ default : -scan -no_update] 3 - border [default =150] 4 - do process [ default = 1 ] 5 - do field list [ default = 1 ]	Runs cataloging of many scans In order to run cataloging with specified camid to be set , use parameter 2 , by calling : cat_nights_scan_new! /ast_dir "-scan -no_update -camid=102 -force_external_camid"
scan_pipe_all!	PARAMETERS: 1 - path to scan directories [ default = /data2/results/ ] 2 - name of night list file [default = night_list ] 3 - place where ast will be placed [ default = current dir ]	runs pi-pipeline on many scans
cat_both_synchro!	PARAMETERS : 1 - list of nights 2 - path to place where catalogs 20050511, 20050512 are 3 - name of database	Important script for running cataloging on already existing ast files. Usage example : nohup ./cat_both_synchro! 20050203.list /scratch/pi10/results/ASAS_PIPELINE/ 20050203new > out 2>&1 &
cat_ast_test!	PARAMTERS : 1 - BORDER=100 2 - N_PARTS=100 ( not used ) 3 - DO_PROCESS=0 ( not used ) 4 - OPTIONS= 5 - AST_LIST=k2a_ast_list 6 - OPT_DB=0 7 - DROP_INDEX=0 8 - CALC_RADEC=0	script for cataloging list of ast files. NOTE : this script cannot run UpdateCatalog ( recalculation ) use cat_single_night! instead
cat_single_night!	PARAMETERS : same as in cat_ast_test!	same as cat_ast_test! but also runs process.sql - to calculate sigma_mag etc ...
cat_single_night_scan!	PARAMETERS : 1 - piaddast2_options=-scan-no_update 2 - border=150 3 - do_process=1	script for cataloging single night of scans

5. Cataloging procedure

Cataloging procedure is described in this document.

6. PI database description

PI database description is described here. In order to run sql login as pi and type pidb!

V/ Other information

1. Usfull links

some imprtant files nessesary to run pi system :
flat frames : http://grb.fuw.edu.pl/pi0/work/data/flat/
asas catalog : http://grb.fuw.edu.pl/pi0/work/data/cat/

PACKAGE NAME	URL	DESCRIPTION
SLAX	http://slax.linux-live.org/ http://www.slax.org/download.php	slackware linux with ssh server and root password
nightfall	http://www.hs.uni-hamburg.de/DE/Ins/Per/Wichmann/Nightfall.html	simulation of binary systems
Analysis - planets ephemeris	http://ssd.jpl.nasa.gov/cgi-bin/eph http://www.strz.uni-giessen.de/~braeun/astrolnk.html , minor planets
IPMI	http://www.comunidelchianti.it/~max/cern/ipmi/node24.html	some information on using IPMI on linux systems
GSC	http://www-gsss.stsci.edu/Catalogs/GSC/GSC1/GSC1.htm http://www.stargazing.net/astropc/download.html	GSC1.2 Star catalog
POSTGRES 8.1	http://www.postgresql.org/ftp/binary/v8.1.4/linux/rpms/fedora/fedora-core-2/	Postgres8 for Fedora Core 2
GCN SITES CFG	http://gcn.gsfc.nasa.gov/sites_cfg.html	GCN sites list
GCN report description	http://gcn.gsfc.nasa.gov/daily_sock_report.html	GCN connection daily report description
log4c	http://log4c.sourceforge.net/	log4c package used for loging pi information to single log file using syslog daemon
ephemeris of planets	http://www.ephemeris.com/ephemeris.php	ephemeris of planets calculated for current date/time
libusb	http://libusb.sourceforge.net/doc/index.html	for USB drivers developers
Mozilla	Mozilla extensions ( TODO list etc )	Nice extensions of mozilla
ssh in java	http://www.astro.umd.edu/ssh/mindtermLaunch.html	ssh client in JAVA
basic astro calculations	http://docs.kde.org/development/en/kdeedu/kstars/ai-colorandtemp.html http://www.ifigure.com/science/astro/astro.htm http://users.zoominternet.net/~matto/Java/index.htm
CCD analysis and information	http://www.astro.virginia.edu/class/majewski/astr511/lectures/detectors/detectors_readout.html http://www.astro.utoronto.ca/~bridge/handouts/ccd.pdf
RAID help	http://tldp.org/HOWTO/Software-RAID-0.4x-HOWTO-4.html http://wiki.gentoo.pl/wiki/Programowy_raid0	information on fixing problems with RAID Generalnie jest tam napisane zeby jak sie robi fsck dysku z raidem to zeby robic na tablicy czyli na /md0 a nie na kazdym kawalku z osobna bo mozna popsuc oba ???!!!
TLE FILES	http://www.io.com/~mmccants/tles/	new location of MCCANTS web page
VOPlot package	http://vo.iucaa.ernet.in/~voi/voplot.htm	package for making astrophysical plots and images ( very usful )
radec -> glon,glat calculator	http://fuse.pha.jhu.edu/cgi-bin/eqtogal_tool	radec -> glon,glat calculator UWAGA : podaje sie RA,DEC a on sam przelicza na galaktyczne !!! nie trzeba tego wogole robic recznie !

Star catalogs :

Catalog Name	URL	DESCRIPTION
GSC	http://archive.eso.org/gsc/gsc

2. Backups

Table below sumarizes were important data is backuped.

COMPUTER	BACKUPED DATA	LOCATION	WHEN
heplx40	/var/www	/backup/www/ NOTE : now in www/html are symbolic links to html.orig and thus I had to change script to backup every catalog separately	daily at 15:00 PM script backup_heplx40.sh
heplx40	pidb database on heplx40	/backup/pidb/ old location : /lhome/piwww/www/pi0/pidb/backup/YYYYMMDD	daily at 15:00 PM script backup_heplx40.sh
heplx40	pidb database on pi2	/lhome/piwww/www/pi0/pidb/backup/pi2/YYYYMMDD
heplx40	other pi settings	/disk01/backup/pi
heplx40	pidb database on pi2	database pidb_backup@heplx40
heplx46	/lhome/*	heplx46:/pidb/lhome/ heplx42':/opt/backup/heplx46/lhome/
heplx49	heplx40 - database, www, pi2 - database	/disk02/backup/
heplx49	system from smok	/disk02/backup/smok/system
heplx42 heplx43	scratch/pi1 z ( heplx41 /dev/hda2 ) system z heplx41 /dev/hda1	heplx42:/opt/backup/heplx41/hdb2 heplx49:/opt/heplx41_hdb2 - exported as /scratch/pi1 heplx43:/disk01/backup/heplx41/hdb1
heplx40	asas database	/disk01/backup/pidb/asas/
heplx49	jmrowca - data from windows computer in 107	/disk02/backup/jmrowca
heplx48	simbad sql file	/opt/pi/dev/pisys/daq/ndir/data/cat/simbad/
pi3	full soft backup	/opt/pi/dev/ARCH/pisys_pi2_20060519_1902.tar
pi3	database backup	/data3/backup/pidb/
pi2	backup of aver20 star catalog seed	/data1/results/picat/aver20
pi2	backup of scan star catalog seed	/data1/results/picat/scan
heplx47	seed scanow	/opt/pi/dev/pisys/daq/ndir/data/backup/scan
heplx47	svn archvie : /opt/pi/svn	heplx47:/disk02/backup/svn/ - backups every night at 5:00 WAW time
heplx46	heplx46:/lhome/	/disk02_hdb3 ( pi1 - SLAVE , scratch pi10 ) heplx49:/disk02/backup/heplx46/heplx46
heplx42	lewhoo - katalog z mgr LWP	heplx42:/disk01/lewhoo/ heplx49:/disk01/lewhoo_MGR/ tempororary also on pi1:/data1/results/lewhoo - soon to be removed
heplx48	pxeclient dla pi2 ( smok w LCO ) /disk02/backup/bootonlan/ /disk02/scripts/ smok2 system and opt ( 20071020 - sobota przed wysylka ! )	heplx48:/disk03/backup/LCO/bootonlan/ heplx48:/disk03/backup/LCO/scripts/ smok2 system + opt : heplx48:/disk03/backup/smok2/20071020
heplx48	gcvs - old version loaded by A.Wrochna + M.Biskup	heplx48:/disk01/backup/gcvs/
heplx48	tycho - old version loaded by A.Wrochna + M.Biskup	heplx48:/disk01/backup/tycho/
heplx48	PLANETOIDS AND PLANETS ephemeris generated for period 2004-2037 by A.Majcher	heplx48:/disk01/majer/planets.sql heplx48:/disk01/majer/planetoids/planetoids.sql
heplx48	Backup of pidb database from pi2 ( pidb @ pi2 ) , mirror jest na pi2 w /data_db/pidb/backup/20080520/ , bardzo wazny backup	heplx48:/disk02/backup/pi2/pidb/20080520/pidb.sql.gz	2008-05-20
heplx40	/var/www/mywiki - piwiki	heplx60:/disk01/backup/heplx40/mywiki	2007-12-12

Events :

EVENT	DATA LOCATION	BACKUP LOCATION	INFO
grb080319b	heplx48:/disk02/results/events/grb/ heplx61:/disk02/results/special/grb/grb080319B/heplx48 heplx43:/opt/pi/dev/pisys/daq/ndir/data/20080318/GRB080319c/NEW/ pi3:/data0/results/20080318 pi2:/data0/results/20080318 pi1:/data1/results/fits/20080318/ Krzywa blasku : /opt/pi/dev/pisys/daq/ndir/data/20080318/GRB080319c/NEW/Event_14.52839333_36.3029_20080319_000000/k2a/parts/mag_list/lc_sorted.txt	heplx61:/disk02/backup/events/grb/grb080319b/	Brigthest GRB ever observed our best observation !!!

Automatic backups :

BACKUPED DATA	DATA LOCATION	BACKUP LOCATION	WHEN / CRONTAB LINE
svn	heplx47:/opt/pi/svn/	heplx47:/disk02/backup/svn/ also in heplx48:/disk01/backup/svn	daily at 5:00 AM see crontab for root : su - crontab -l 0 5 * * * rsync -auv /opt/pi/svn/ /disk02/backup/svn/
www	heplx40:/var/www/	heplx40:/backup/www/	daily at 15:00 PM ( see crontab for root at heplx40 )
pidb ( heplx40 )	heplx40:/db/pidb/	heplx40:/backup/db/pidb/	daily at 15:00 PM ( see crontab for root at heplx40 )
heplx40	heplx40:/lhome/ heplx40:/etc/	heplx40:/backup/sys/	daily at 15:00 PM ( see crontab for root at heplx40 )

BACKUPS IN LCO and of LCO

BACKUPED DATA	DATA LOCATION	BACKUP LOCATION
cron	crontab -l	/opt/pi/dev/pisys/backup/
InterestingObjects table	aver20_2006@pi3	/opt/pi/dev/pisys/daq/ndir/data/cat/io/backup
system on pi2	/	/data2/backup/system/ script : nohup backup_system_partition2! > out 2>&1 & also /dev/sdb1/ - but now it is system partition since crash of SCSI disk
table interestingobjects from aver20_2006	aver20_2006@pi3	heplx43:/opt/pi/dev/pisys/daq/ndir/data/cat/io/aver20_2006_at_pi3
star catalog aver20_2006 ( on 20070516 )	aver20_2006@pi3	heplx48:/disk03/results/PI_PIPELINE/aver20_2006 and in database aver20_2006@heplx48
pi2 database ( /data2/pidb/ )	pi2:/data1/pidb/	This is database till 20070511 - until SCSI disks were working, after this moment database was moved to /data2/pidb/ due to disk access efficiency problems

BACKUPY SYSTEMOW Z LCO ( pi1 , pi2 , pi3 on 20071116 )	heplx69:/data4/sys/	Backupy wszystkich systemow z LCO na dzien sprzed wyjazdu GW ( 2007-11-16)

Star catalog backups

TODO : przeniesc backup z heplx40 na inny komputer !!!! ( nie publiczny heplx40 ! ) na razie sa tez na :

root@heplx69:/disk3_data/backup/20070917

BACKUPED DATA	DATA LOCATION	BACKUP LOCATION
2004_2005 star catalog	2004_2005@heplx40 ( public ) 2004_2005@heplx61 ( dev ) oraz awaryjna na 2004_2005@heplx49	heplx61:/disk02/backup/pidb/2004_2005/2004_2005.sql (132GB) - dump after Kasia'a deep_purple done on 2007-12-07 heplx40:/backup/db/2004_2005/2004_2005_old.sql ( 130 GB ) - OLD-ORIGINAL BEFORE KASIA RUN deep_purplne criteria on it smok2 = heplx69:/disk3_data/backup/2004_2005/2004_2005.sql ( 130 GB ) backup pelny ( juz po CLUSTER ) : ( byl w heplx40:/2004_2005/backup/20070916/ ) heplx42:/opt/backup/heplx40/pidb/2004_2005/20070916/
2006_2007 star catalog	2006_2007@heplx61 ( dev ) 2006_2007@heplx40 ( public ) 2006_2007@heplx42	heplx61;/disk02/backup/pidb/2006_2007/2006_2007.sql ( 114 GB ) heplx42:/disk01/backup/2006_2007/ heplx40:/2004_2005/backup/2006_2007/
asas catalog	pidb_asas@helx40	heplx40:/backup/db/asas
gcvs catalog	gcvs@heplx40 ( gcvs@heplx48 )	heplx40:/backup/db/gcvs/
tycho-2 catalog	tycho@heplx40 ( tycho@heplx48 )	heplx40:/backup/db/tycho/

TEMPORARY DATA :

BACKUPED DATA	DATA LOCATION	BACKUP LOCATION
scan nights 20061124 20061125 20061126 20061127 20061128	some nights to be given to student for tests	heplx48:/disk01/results/scan_data/

3. Usfull linux / fedora hints :

Running system in single mode :
after grub menu apears press e and add 1 add the end of line of parameters passed to kernel
Linux rescue :
put Fedora DVD or rescue disk and after booting from cdrom enter : linux rescue , press Enter and
answer some obvious questions, after all it is posible to check disks etc ...

4. Usfull downloads

WHAT	COMPUTER	LOCATION
fedora core2 dvd - iso	heplx48	/disk01/download/fedora/2
fedora core4 dvd - iso	heplx48	/disk01/download/fedora/4 /pi20/msok/fedora/4/
fedora core5 dvd - iso	heplx42	/pi20/msok/fedora/5/
fedora core5 - cdrom x 5	heplx43	/pi20/msok/fedora/5/cdrom/
SLAX Server Edition v 5.1.7b	heplx43	/pi20/msok/download/slax/
Visual studio deployment		http://msdn2.microsoft.com/en-us/library/ms235316.aspx http://blogs.msdn.com/aymans/archive/2006/04/04/568466.aspx
knoppix	heplx43	heplx43:/disk01/download/knoppix/

5. External catalogs

There multiple external catalogs loaded to pi format database these are :

tycho
asas
gcvs

Also I downloaded GSC catalog it is available on heplx43, by doing :
cd /pi20/msok/cat/gsc/gsc1.2/src
./gsc.exe -c 23.23/-19.23 -h -r 10

27. Special Operations and Fixes in system ( click here )

To be continued ....

VI/ LCO system

1. External software

PACKAGE NAME	WWW LINK	DESCRIPTION
vgrabbj	http://vgrabbj.gecius.de/	Program from taking photos with webcam camera Philips installed in asas dome and connected to pi2 machine. Currently used version is 0.9.6 and currently ( 20060529 ) it is the highest one

Please send questions and remarks to : msok@fuw.edu.pl