| 
  • If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • You already know Dokkio is an AI-powered assistant to organize & manage your digital files & messages. Very soon, Dokkio will support Outlook as well as One Drive. Check it out today!

View
 

kmax2ntpl

Page history last edited by jgibelin 14 years ago

 

kmax2ntpl

 
 

This Fortran/C/C++ set of routines convert a data file written with KMAX to virtually any other data format (screen, ascii, ROOT, CERNLIB, ...).

Started and maintained by J. Gibelin (gibelinunicaen.fr).

For information send me an email.

For debugging please include in your email at least the last (part of the) log file and information on your system (the output of uname -a for example).

 

Status

 

I presently left Laurence Berkeley National Laboratory, where I started working on this project.

 

If -- or course -- I will be happy helping you installing this program on your computer I can not guaranty  that I will have time and resources to update or modify it.

Please be also aware of the fact that I am presently not leaving in USA and it can take some time (jet lag !) before I answer your email (to be preferentially sent to gibelinunicaen.fr).

 

However, this project is very general: it answers the question "How can I read this raw data in a humanly readable way". There is thus a big chance that I will get back to this code sometime in the future for a different project. I will of course post any change on this page.

 

 

Download

 

You can download the latest version here:

 

*kmax2ntpl.ver0.9982.tgz (Saturday Jan 24th 2009)

 

Older versions (beware: they might be buggy):

 

*kmax2ntpl.ver0.9981.tgz (Wednesday May 29th 2008)

*kmax2ntpl.ver0.998.tgz (Wednesday May 29th 2008)

*kmax2ntpl.ver0.9971.tgz (Wednesday May 28th 2008)

*kmax2ntpl.ver0.997.tgz (Wednesday Sep 26th 2007)

*kmax2ntpl.ver0.9961.tgz (Thursday Sep 20th 2007)

*kmax2ntpl.ver0.9951.tgz (Tuesday July 31st 2007)

*kmax2ntpl.ver0.9931.tgz (Wednesday June 13th 2007) 

*kmax2ntpl.ver0.993.tgz (Wednesday June 6th 2007)  

*kmax2ntpl.ver0.992.tgz (Thursday May 24th 2007)  

*kmax2ntpl.ver0.9911.tgz (Monday May 22th 2007)

*kmax2ntpl.ver0.991.tgz (Wednesday April 25th 2007) 

*kmax2ntpl.ver0.99.tgz (Friday April 17th 2007)

*kmax2ntpl.ver0.982.tgz (Friday March 23rd 2007) 

*kmax2ntpl.ver0.981.tgz (Thursday March 14th 2007) 

*kmax2ntpl.ver0.98.tgz

*kmax2ntpl.ver0.97.tgz

*kmax2ntpl.ver0.96.tgz

 

 

Installation

 

Download the latest version above.

 

Unpack the archive somewhere in an appropriate directory:

tar zxvf kmax2ntpl.verX.XX.tgz

Compile the source:

make

If you want to install the executable in the path (need root password) as well as the manual:

make install

If you want to read the manual installed in the current directory:

make man

This program as been tested with Fedora Core 5, 6, iBook and PowerPC and gcc 3.4.3 and higher.

The makefile should handle the Mac/PC portability. May not work with linux kernel 2.4 and older.

 

You must modify your makefile to compile and links with CERNLIB (ROOT is done automatically if ROOT is installed, see bellow).

 

CERNLIB and PAW : 

 

Technically speaking you don't need either root or cernlib to compile kmax2ntpl.

However, this program has been written essentially has an interface between raw data and CERN libraries (root or paw).

I thus strongly suggest you install ROOT, from the sources (see root.cern.ch) following the procedure described in

root.cern.ch/root/Install.html). I also suggest, if you have super-user privileges, to follow the "fixed location" procedure.

This will grandly simplify your (and other users) access to ROOT and the kmax2ntpl's Makefile will normally work by itself.

 

Note that I systematically installed ROOT and thus the list of OS/Computer where kmax2ntpl works also apply to ROOT.

 

 

Man page

 

NAME

KMAX2NTPL - Convert Kmax event files to other formats.

SYNOPSIS

kmax2ntpl [options] input_struc_file input_event_file(s) directory|output_ntuple_file

DESCRIPTION

 

 Convert KMAX event files to other formats.  Needs as input a file describing the data structure and the relation between observable and electronic modules (input_struc_file, see below for details).

 

 It needs KMAX data file(s) as input (input_event_file). You can specified as many files as you want, kmax2ntpl will process them in the order you gave them to it. From august 2007 you can also read data files that have been compress by removing the 16bits zeros or by gzip. Note that reading gziped file is (about twice) slower.

 

 By default the program output to a CERN, ROOT TTree formatted file, named from the body of the first input_event_file with a *.root as extension. You can change the output format with the options described below and the name of the output by specifying a output_ntuple_file. Note that an output directory or output file name are now mandatory (due to the possibility of reading more that one data file).

 

OPTIONS

-help:
shows the help
-v:
verbose
-list:
lists all electronic modules defined in the code and some help
-2bytes:
input data file is 2 byte size (compressed) this flag is automatically set if the data file extension is *.cevnt instead of *.evnt
-ascii:
output in ASCII file, with as default a *.txt extension.
-screen:
output on screen
-csv:
output in Comma Separated Value file format, with as default a *.csv extension.
-zeros:
output data w/o zero suppress (-ascii or -screen flag only)
-root:
output in root tree file format, with as default a *.root extension.
-ntpl:
output in PAW ntuple file format.
-scaler=s:
scaler output status:
if s=yes  sort and output the scaler data (default)
 if s=only sort and output the scaler data only
 if s=no   do not sort or output the scaler data
-map:
create an additional histogram that maps extended channel.

 Warning: can substantially slow down the program

-cal=file:

 use the parameters in file to calibrate the data

-calonly, -noraw:
do not convert raw data. Calibrated only. Option to save hard drive space.
-%=char:
 replace the default '%' file separator by 'char'

 useful when you renamed your data files to be readable by another program (for example TSCAN) and want to read them back with kmax2ntpl. 

-events=n:
limit the number of read events to n.
If n is an integer: read from 0 to n. 
 Otherwise, if n is a range n0-n1 read from event n0 to event n1.
 Thus -events=0-100 is equivalent to -events=100.
Warning: no space after and before the `=' sign.
-debug:
equivalent of -screen -zeros -v
-log:

 redirect all errors (even the most improbable ones) and append infos to kmax2ntpl.log.

 This file has to be manually removed.

-allerror:
print all encountered errors at the end.
-f --force:
do not prompt before overwriting.
-listerr:
list all defined error flags. Since an array of error is automatically generated and saved with the nutple (TTree only for now) this flag helps to assign which array is which error.
-test:
intermediate debugging
-xinput:
test the input file. Output result and exit.

DATA STRUCTURE FILE

The data structure file should be organized in four columns
Column 1:
observable label, limited to 128 different ones
Column 2:
observable index
Column 3:
electronic device name. Case insensitive. Use option --list to obtain all defined modules.
Column 4:
If Column 5 is 0 or something else than an integer then Column 4 corresponds to the cumulative channel. Typically VSN*..+... Otherwise Column 4 is the VSN number. Starts from 1.
Column 5 (optional):
Channel number (starts from 1) for the corresponding module of VSN given in Column 4. Then the programs calculate automatically the cumulative channel number.
Warning: if this column is not a integer greater than 0 then the program takes the number in Column 4 as the cumulative channel number.
Warning: maximum line length: 256 characters
Use comma, space or pipe | to separate columns. You can check that the file is properly read by using the verbose -v flag or better the -xinput.
A Comment line
can be added by starting the line with a double slash //. Everything added to a properly defined normal line will be ignored, hence you can use this space as comment. No need to use // although it is recommended.
Scaler Data
are treated differently by the sorting routine (different format, ntuple, ...) but for simplicity its declaration is identical to a normal electronic module. No array is allowed thus column 2, mandatory, is however meaningless.
Dead channel
For safety and modularity you have to declared all the observable of an array. However, its corresponding electronic channel might be dead, unplugged, etc. You can declared a dead channel by putting a least one start * in column 3 followed by any number in column 4.

Example

The example below named test.inp defines some silicon observable. The first line is a comment. The second line tells kmax2ntpl that the first channel of the Silena electronic module corresponds to the first index of an energy observable array E. The third line that the second channel of the Silena corresponds to the second index of the array E. Then comes a comment and the next four lines correspond to Ge clover electronic and retrieve the energy observables into a four index array Eg. However the 3 index is a dead channel and is notified by starts. The last lines give the name RawMG and GatedMG to the first two scaler channels. Note that RawMG and GatedMG are not and can not be array and are thus followed by 0 in column 2.

----------------------------------------------

 

 // Definition for the silicons, E

 E  1, Silena 1 // This is the first Silena channel

 E  2, Silena 2 // Here is the second

 // Definition for the Ge

 Eg 1, Clover 1 // And the clovers

 Eg 2, Clover 2 

 Eg 3, *** 3 // Dead channel 

 Eg 4, Clover 4

 // Scaler definition

 RawMG 0, SCALER 1 

 GatedMG 0, SCALER 2

----------------------------------------------

If you want to use the Column 5 then the input file will equivalently looks like:

----------------------------------------------

 

 // Definition for the silicons, E

 E  1, Silena 1 1 // This is the first Silena channel

 E  2, Silena 1 2 // Here is the second

 // Definition for the Ge

 Eg 1, Clover 1 1// And the clovers

 Eg 2, Clover 1 1 

 Eg 3, *** 1 3 // Dead channel 

 Eg 4, Clover 1 4               

 // Scaler definition

 RawMG 0, SCALER 1 

 GatedMG 0, SCALER 2

----------------------------------------------

 

CALIBRATION OF DATA

You can also roughly calibrate your data. In order to do this, prepare a file (by default with the extension *.prm) with your calibration parameters. This file looks like the structure file one but instead of putting the electronic and channel module for a given observable you declare the three coefficient (a,b,c in the following) of a quadratic calibration i.e. the relation between calibrated data (CAL) and raw data (RAW) is:

 

CAL = a + b*RAW + c*RAW^2

 

You can furthermore specify the units by putting them in comment at the end of the line in square brackets [ ].

Example

Here is a example following the previous input file:

----------------------------------------------

 

 // Definition for the silicons, E

 E  1 116.039864 3.6317567 0. // [keV]

 E  2 169.546674 3.5753880 0. // [keV]

 // Definition for the Ge

 Eg 1 -7.53521085 0.54003370 -0.00000048 // [keV]

 Eg 2 -11.0588063 0.53641445 -0.00000054 // [keV]

 Eg 3 0. 1. 0. // [keV]

 Eg 4 0.193756447 0.49586115  0.00000030 // [keV]

----------------------------------------------

where the E array is linearly calibrated around 3keV/ch and the clover about 0.5keV/ch with a quadratic correction. Note that scalers can be calibrated too.

 

HACKING

This code is of course open source and you are welcome to modify it. Better that this I tried to wrote it in a way it can be modified. It it essentially written in C but for compatibility with CERNLIB and CERN ROOT libraries some part of the code are written in Fortran or C++. You have one main file kmax2ntpl.c where you can find all the modules and functions described in the kmax2ntpl.h header file. The sub_2root.cxx contains all the routine to create and fill a CERN ROOT TTree. All functions and routines are documented. For basic hacking you might just want to know: how to add another unpacking routine for a new electronic modules and perhaps how to add a new ntuple format.

 

 

Code structure

Essentially the structure of the code is the following:
1) Read flags, read the data input file, check the number of sub data files, and look for the headers.
2) Create the corresponding events arrays and generate the first ntuple file according to the name given.
3) For each events unpack the data and pass them to the corresponding ntuple filling routine.
4) Close everything and generate a error report.

Adding a new unpacking routine

As to date 4 routines as been written for unpacking Silena ADC 4418, ORTEC AD413, FERA ADC and TODD Modules. There are respectively named read_Silena, read_ADC413, read_FERATDC and read_Clover. They take as argument the address of the corresponding data header. To had a new module just create a new function read_newmodulename. The corresponding data should be attached to a given kind of header and the module name extracted with the function whichmodule -- which must hence be modified. The call to the corresponding read_newmodulename function, driven by the corresponding result of whichmodule is located in the SortData function.

 

Adding a new ntuple format

This section of the manual should be improved but essentially there is three kind of routines that must be created and called: a initialization, a routine to fill the ntuple and one to end the writing. For now please use the examples of the PAW NTUPLES and ROOT TTree respectively called init_ntuple_file, fill_ntuple_file, end_ntuple_file and init_TTree_file, fill_TTree_file, end_TTree_file (these last three being located in the sub_2root.cxx file).

 

PRECOMPILATION FLAGS

The Makefile provided should take care of the essential compilation flags. Users must however chose between two compilation brands: with (DEBUG = 1) or without (DEBUG = 0) debugging routine. In addition other precompilation flags (essentially for the DEBUG = 0 option) can be (un)defined in order to change the code performance (see also the Makefile):

 

-D__FAST_FILLING

This option select a different code for filling the unpacked data structure. Without it all the structure is scanned and the row of interest is filled. With it: a map of the position of the row of interest in the unpacked data structure is generated and this row is directly filled. Can be dangerous if the mapping fails due to corrupted data. Improves speed by a factor 3.

 

-D__NO_ERROR_REPORT

With this flag no error is reported. The speed improvement may depend on the machine since checking for error often result in loop breaking.

 

TODO

Now it is pretty much working but I should clean. What about creating two dimension observable arrays?

 

BUGS

Well... this being a beta version ... But please if you think you found a bug or want/made some improvements send me a mail: jdgibelinlbl.gov.

Note: if you found a bug please include in your email at least the last (part of the) log file and information of your system (the output of uname -a for example).

 

 

 

 


 

 

This document was created by man2html, using the manual pages.

 

Time: 18:58:25 GMT, September 26, 2007

Comments (0)

You don't have permission to comment on this page.