TESTING FLASH RECOGNITION ALGORITHMS


1. Testing plane tracks on summed frames

I was doing  tests of algortihms  acting on 8 summed frames. It is importatnt to use so called "plane track" algorithm in this part, as
this is finding tracks using many points on single frame ( summed frames have "lines of events" ) also velocity is not checked in this
algorithm.
I have frames summs at heplx43:/disk01/results/20040929_sum8 and flash recognition algorithm may be run on this data , there
was new paramter added to indicate that this is such type of algorithm it is :

             CCD_ON_SUMED_FRAMES=8

8 - indicates that analysis is performed on 8 averaged frames. This can be done that first single frames are averaged using pi_red_frame program or
in order to run analysis on single frames and make average of several frames use parameter :

           CCD_AUTO_CALC_SUM=1

in order to run analysis just type :

         nohup ccdddouble > out 2>&1 &

Other important parameters to be set are :

PARAMNAME
DESCRIPTION
 GOOD VALUE
CCD_CHECK_TRACKS
 turn off normal tracks finder
 0
CCD_CHECK_PLANE_TRACKS
 enable plane tracks finder
 1
CCD_MAX_CHI2_IN_PLANE_TRACK
 max chi2 for track seed on 3 points
 2 - will be verified
CCD_MAX_CHI2_IN_PLANE_TRACK_TO_OLD
 max chi2 for new points to be added to existing track
 2 - will be verified
CCD_NUM_BACK_FRAMES_FOR_PLANE_TRACK
 number of back frames from which events are used to form
track
 10 - to be verified
CCD_MIN_FRAMES_ON_PLANE_TRACK
 minimal number of frames with events to fit track
 0
CCD_NEW_LAPLACE_TRESHOLD_IN_SIGMA
 Tn treshold
TO BE CHECKED by LWP
CCD_MAX_LAPLACE_ON_OTHER_IN_SIGMA
 Tv treshold
 TO BE CHECKED by LWP
CCD_MIN_POINT_NO_ON_PLANE_TRACK
 Minimal number of events in track
5 - verify
CCD_USE_FOUND_POSITION
 uses m_Point instead of m_MaxPoint to calculate RA,DEC
and fits tracks - this can be good to have many points for
track with IF_MORE cut disabled ( CCD_SKIP_IF_MORE_THEN=0 )
NOTE : this may cause some lose of efficency due to coincydence if
star is identified on oposite sites of PSF on both cameras coincydence
may fail !!!
 		0
CCD_MAX_NUMBER_OF_EVENTS_AFTER_COIC
 limit on number of events after coincydence ( in case of no if_more cut it should be increased to 150 or more ) but too much is not good either
 150
CCD_REJECT_SINGLE_EVENTS
 rejection of all events found in single frames analysis , this is good
I assume that all interesting events were already found there
( satellite rejection is better there - no efficiency loss )
1
CCD_ALWAYS_REREAD_SINGLE_FRAME_EVENTS
 if re-read single frames events every time new frame is analysied or assume that single frames analysis is finished and first read is complete
 0
CCD_REJECT_SINGLE_TRACKS
 rejection of tracks identified in single frame analysis ( if saved to database )
 1
CCD_ALWAYS_REREAD_SINGLE_FRAME_TRACKS
 if re-read single frames tracks every time new frame is analysied or assume that single frames analysis is finished and first read is complete 0

Parameters for this analysis are grouped in cfg file online_sum8.cfg ( see $SRCDIR/ccd/cfg/WZORY_CCD_CFG/doubleCAM_Real/2K2K_2K2K )
The first thing I checked were tracks , I enabled showing of chi2 on 3 points by setting CCD_SHOW_CHI2_3POINTS=1 , after using script
get_chi2_3points! I obtained distribution ( see plots in different chi2 ranges  : 0_100, 0_10, 0_5, 0_1, more plots ) , however I generated
10 points per frame with program gen_chi2 and checked generated chi2 distribution and it is very similar - 0_100, 0_10, 0_5, 0_1 ( more plots ) - so finally I left chi2_treshold=2.0 !!! There is a peak at chi2 ~= 4.5 in data so maybe there is something there ... , I will check distribtuion of chi2 of tracks with >5 points for example

Distribution of chi2 for tracks with >=5 points ( but also failed tracks - which were tried to have 5th point ) is show here ( 0_500, 0_5000, 0_50000, 0_10000, eps and more )
So here I decide  that  cut on chi2 can be set on value  ~= 2000 with requirement that  n_points_track>=5  ( CCD_MIN_POINT_NO_ON_PLANE_TRACK=5 ) .
I checked same distribution on the points generated by "monte carlo" program :
          gen_chi2 100000 30 200.  > out_very_big_chi2_200 2>&1 &
Total number of generated points is : 2800000
And obtained the following distributions of chi2 for >=5 points in track , >=6, >=7 and >=8 points in track
So this gives the following probabilities for accidental track to be fitted with chi2/point <=100 :

N_POINTS
 PROBABILITY TO  FIT FALSE TRACK - upper limit
5
 0.0004
6
 0.00002
7
 0.0000014
8
 ~ 0.00

So it is quite safe to put cut on CHI2=100 and minimum number of points in track = 5
( heplx43:/disk01/results/20040929_sum8/TESTY/TRACKS/chi2_vs_points/mc )

Without velocity checking CCD_MIN_EVENTS_NO_IN_TRACK=3 is to loosy cut, I will change it to 4 or 5 , more probably to 5
Checking chi2 for tracks with >3 points ( enable CCD_SHOW_NPOINTS_VS_CHI2=1)

There is also a chance that due to averaging track will disapear , however program can events in tracks found by single frame analysis algorithm, so
those could be put to database to provide convinient way to do so .

Tests performed at heplx43:/disk01/results/20040929_sum8/TESTY/TRACKS/

Dla takich duzych trackow typu samolotowego problemem jest jednak to ze czesto wiekszosc eventowow wypada na koincydencji i zostaje tylko kilka
- no chyba zeby moze wylaczyc If_more_cut wtedy moze byc tego wiecej to tu juz chyba Lecha zapuszcze na to :

chyba Tn=2.0 to za malo sprawdzam z Tn=2.0

Wylaczylem na proble if_more oraz urzywanie pozycji m_Point zamiast max w clustrze - zeby lepiej tracki fitowac, poza tym ze wzgledu na
brak if more trzeba bylo zwiekszyc limit na # przypadkow :

2. Dla Lecha :
 

Dane w heplx43:/disk01/results/20040929_sum/

lista k2a_aver_list, k2b_aver_list oraz plik cfg zaincludowac na poczatku : heplx43:/opt/pi/dev/pisys/daq/src/ccd/cfg/WZORY_CCD_CFG/doubleCAM_Real/2K2K_2K2K/online_sum8.cfg

3. Flash recogintion algorithms details

In coincydence mode new events are found on both cameras, and for each event cluster is found and coordinate of cluster center of mass
is assigned to event. Overlaping events from both cameras are removed, so in case there is plane track only one single pixel ( ~max pixel )
will be taken to list of events for conincydence - this I have to think about, for sumed frames it maybe better to keep many points and
have them to fit track !!! - I will check how to do it


4. Functions and procedures


Function Name
 DESCRIPTION
CCD_Analyser::AnalyseNewFrameOpt
main function, finds events on single camera, calls specific algorithm depending on paramters
CCD_Analyser::AnalyseNewFrameWithAverageOfPrevAlg
 function analysing new frame in comparison to average of several previous frames





5. Converting fits -> html :

- poprawic analize na posumowanych - niech ten sam ccddouble dziala,
  ale tak ze jakis pi_red_frame dziala robi redukcje i zapisuje na dysk
  sumy a ten je wczytuje : run_online_sum8!
        ale trzeba by poprawic jeszcze robienie wycinkow z pojedycznych klatek
  w skrypcie : cppi2_online! zmienic :
                prepOnlineSum! -> prepOnlineSumNew!
  dorobic wywolanie skryptu w cronie :
                run_online_sum8!
  zapisywac do SumEvents - albo linka zrobic symbolicznego ?
  SumEvents
  get_single_events_from_sum! SINGLE_FITS_DIR_PATH/
  dodac tylko opcje do getpart2 zeby tworzyl pliki :
        eventframe2188_camera0_eventno4_currframe2202_02202.fitc
        dla obu kamer camera0
  testy : cd /disk01/msok/tmp/sum8/SUM/ONLINE_ANALYSIS/
     get_single_events_from_sum! /scratch/pi_fits2004/results/fits/20040929/
     cd /disk01/msok/tmp/sum8/SUM/ONLINE_ANALYSIS/RESULTS/20060413_172206
     nohup ./do_singles! > do_singles.out 2>&1 &

6. Problems with sum8 algorithm :