Search below for 'logbook_sop' for help on usage.

Sections

• AMANDA - 417
•   Computing - 183
•   Hardware - 103
•   Misc - 2
•   Runs - 57
•   Supernova - 39
•   TWR - 7
•   Temperature - 5
•   Training - 6
• IceCube - 1932
•   AURA - 14
•   Commissioning - 81
•   DAQ - 15
•   DOMs - 66
•   Deployment - 12
•   HoseReel - 5
•   ICL - 179
•   IceTop - 114
•   Inclinometer - 9
•   Infrastructure - 128
•   Misc - 220
•   PBS - 2
•   PNF - 17
•   Power - 15
•   Runs - 718
•     RCAs - 297
•   SPATS - 3
•   SPOTS - 14
•   SPS - 285
•   StandardCandle - 2
•   Testing - 8
•   Training - 5
• MAPO - 124
• Misc - 41
• PowerOutages - 1
• SPADE - 1140
• SPASE - 6

Search

Advanced Search

Archives

November 2009
Sun	Mon	Tue	Wed	Thu	Fri	Sat
1	2	3	4	5	6	7
8	9	10	11	12	13	14
15	16	17	18	19	20	21
22	23	24	25	26	27	28
29	30

• November 2009 (61)
• October 2009 (52)
• September 2009 (59)
• August 2009 (57)
• July 2009 (60)
• June 2009 (61)
• May 2009 (66)
• April 2009 (52)
• March 2009 (81)
• February 2009 (30)
• January 2009 (85)
• December 2008 (118)
• November 2008 (136)
• October 2008 (143)
• September 2008 (116)
• August 2008 (219)
• July 2008 (228)
• June 2008 (269)
• May 2008 (236)
• April 2008 (160)
• March 2008 (145)
• February 2008 (141)
• January 2008 (131)
• December 2007 (104)
• November 2007 (103)
• October 2007 (8)
• September 2007 (9)
• August 2007 (19)
• July 2007 (10)
• June 2007 (23)
• May 2007 (15)
• April 2007 (34)
• March 2007 (83)
• February 2007 (17)
• January 2007 (3)
• December 2006 (14)
• November 2006 (42)
• October 2006 (40)
• September 2006 (37)
• August 2006 (30)
• July 2006 (57)
• June 2006 (27)
• May 2006 (40)
• April 2006 (50)
• March 2006 (56)
• February 2006 (38)
• January 2006 (93)
• December 2005 (4)

Apr 29, 2008

Starting combined runs

---

By: TeX
Time: 2008120 0453Z
Music: 80's pop mix

0453Z	By MK's command, I am going to attempt to bring up a combined run
	as ethan is not available.

	# let's stop the standalone TWR run:
	[twrdaq@ic-twrdaq00 ~]$ date
	Tue Apr 29 04:53:50 UTC 2008
	[twrdaq@ic-twrdaq00 ~]$ cd current_daq/
	[twrdaq@ic-twrdaq00 current_daq]$ ./stop_twr_daq.pl
	   running process [eventbuilder] found with id [27074]
	   KILLING eventbuilder process with id [27074] with signal [INT]
	   TWRDAQ stopped softly.


	# Now the standalone i3only followed by a combined run:
		pdaq@sps-expcont[~] 04:53:42 (0)$ anvil stop sam
		'sam' Stopped
		pdaq@sps-expcont[~] 04:55:30 (0)$
		pdaq@sps-expcont[~] 04:55:43 (0)$ ./starti3am
		Sub-system 'daq' are now under control of anvil.
		Sub-system 'twr' are now under control of anvil.
		2008-04-29 04:55:48: Started 'sam' service with PID=32534
		2008-04-29 04:55:48: Parameters for this run set:
		2008-04-29 04:55:48:           Run Mode: PhysicsTrig
		2008-04-29 04:55:48:          DAQ Label: sps-IC40-massive-icetop-changes-V043
		2008-04-29 04:55:48:          JeB Label: MergeEvents (1)
		2008-04-29 04:55:48:          PnF Label: PhysicsFiltering (1)
		2008-04-29 04:55:48:         Run Length: 28800
		2008-04-29 04:55:48:     Number of Runs: 40000
		2008-04-29 04:55:48: Started 'run' service
		2008-04-29 04:55:48: Parameters for this run:
		2008-04-29 04:55:48:         Run Number: 110908
		2008-04-29 04:55:48:           Run Mode: PhysicsTrig
		2008-04-29 04:55:48:          DAQ Label: sps-IC40-massive-icetop-changes-V043
		2008-04-29 04:55:48:          JeB Label: MergeEvents (1)
		2008-04-29 04:55:48:          PnF Label: PhysicsFiltering (1)
		2008-04-29 04:55:48: Starting run 110908
		2008-04-29 04:56:00: Starting 'twr' sub-system
		2008-04-29 04:56:00: Starting 'daq' sub-system
		2008-04-29 04:57:04: Sub-system 'daq' is in 'Error' state, not 'Starting'
		2008-04-29 04:57:04: Successive daq failure number 1
		2008-04-29 04:57:04: Stopping run 110908
		2008-04-29 04:57:10: Stopping 'twr' sub-system
		2008-04-29 04:57:10: Recovering 'daq' sub-system
		2008-04-29 04:57:20: Stopped run 110908
		2008-04-29 04:57:21: Stopped 'run' service
		2008-04-29 04:57:22: Started 'run' service
		2008-04-29 04:57:22: Parameters for this run:
		2008-04-29 04:57:22:         Run Number: 110909
		2008-04-29 04:57:22:           Run Mode: PhysicsTrig
		2008-04-29 04:57:22:          DAQ Label: sps-IC40-massive-icetop-changes-V043
		2008-04-29 04:57:22:          JeB Label: MergeEvents (1)
		2008-04-29 04:57:22:          PnF Label: PhysicsFiltering (1)
		2008-04-29 04:57:22: Starting run 110909
		2008-04-29 04:57:33: Starting 'twr' sub-system
		2008-04-29 04:57:34: Starting 'daq' sub-system
		2008-04-29 04:58:39: Sub-system 'daq' is in 'Error' state, not 'Starting'
		2008-04-29 04:58:39: Successive daq failure number 2
		2008-04-29 04:58:39: Stopping run 110909
		2008-04-29 04:58:44: Stopping 'twr' sub-system
		2008-04-29 04:58:44: Recovering 'daq' sub-system
		2008-04-29 04:58:54: Stopped run 110909
		2008-04-29 04:58:55: Stopped 'run' service
		2008-04-29 04:58:55: Started 'run' service
		2008-04-29 04:58:55: Parameters for this run:
		2008-04-29 04:58:55:         Run Number: 110910
		2008-04-29 04:58:55:           Run Mode: PhysicsTrig
		2008-04-29 04:58:55:          DAQ Label: sps-IC40-massive-icetop-changes-V043
		2008-04-29 04:58:55:          JeB Label: MergeEvents (1)
		2008-04-29 04:58:55:          PnF Label: PhysicsFiltering (1)
		2008-04-29 04:58:55: Starting run 110910
		2008-04-29 04:59:07: Starting 'twr' sub-system
		2008-04-29 04:59:07: Starting 'daq' sub-system
		pdaq@sps-expcont[~] 05:00:13 (0)$ anvil stop sam
		'sam' Stopped

		(dash) Waiting for stringHub#0
		(dash) Waiting for stringHub#0
		(dash) Waiting for stringHub#0
		(dash) Waiting for stringHub#0
		(dash) Failed to start run: RequiredComponentsNotAvailableException("Still
		waiting for stringHub#0") in waitForRequiredComponents() (DAQRun.py:240) <-
		build_run_set() (DAQRun.py:371) <- run_thread() (DAQRun.py:586)
		(dash) TypeError("cannot marshal None unless allow_none is enabled") in
		dump_nil() (xmlrpclib.py:620) <- __dump() (xmlrpclib.py:616) <- dumps()
		(xmlrpclib.py:604) <- dumps() (xmlrpclib.py:961) <- __request()
		(xmlrpclib.py:1310) <- __call__() (xmlrpclib.py:1029) <- ?() (:0) <-
		rpccall() (DAQRPC.py:64) <- rpc_daq_summary_xml() (DAQRun.py:859)
		(dash) Recovering from failed run 110910...



	Must need a DAQLaunch I bet! Hmm, DAQLaunch.py returns but nothing
	running on sps-amanda, must need to deploy too!

	pdaq@sps-build64:DeployPDAQ.py -c ic40-it6-info

	


0507Z	As pdaq@sps-expcont:
		DAQLaunch.py
			errors about multiple ones running, killed this.
	
		DAQLaunch.py -k
		DAQ is not currently active
		pdaq@sps-expcont[~] 05:06:19 (0)$ DAQStatus.py
		CNC localhost:8080
		-----------------------
		103 unused components
		idle
		  2 amandaTriggers: 0 0
		  2 eventBuilders: 0 0
		  2 globalTriggers: 0 0
		  2 iceTopTriggers: 0 0
		  2 inIceTriggers: 0 0
		  2 secondaryBuilderss: 0 0
		  91 stringHubs: 0 21 21 29 29-30 30 38 38-39 39-40 40 44 44-45 45-46 46-47
				 47-48 48-49 49-50 50 52 52-53 53-54 54-55 55-56 56-57 57-58
				 58-59 59-60 60-61 61-62 62-63 63-64 64-65 65-66 66-67 67-68
				 68-69 69-70 70-71 71-72 72-73 73-74 74-76 76-77 77-78 78
				 81-82 82-83 83-84 84-85 85-86 86
		-----------------------
		0 run sets
		DAQ state is STOPPED


	Still getting output to screen so:
	ps auxww | grep DAQ
	pdaq      8655  0.0  0.1 113564 9600 pts/2   Sl   05:02   0:00 python /usr/local/pdaq/pDAQ_current/dash/DAQRun.py -r -f -c /usr/local/pdaq/pDAQ_current/config -l /mnt/data/pdaq/log -s /mnt/data/pdaqlocal -u ic40-it6 -a /mnt/data/pdaq/log-copies -n
	pdaq      8682  6.0  0.0 99476 7852 pts/2    Sl   05:02   0:19 python /usr/local/pdaq/pDAQ_current/dash/CnCServer.py -l localhost:9001
	pdaq     12487  0.0  0.0 51084  668 pts/2    S+   05:07   0:00 grep DAQ
	pdaq@sps-expcont[~] 05:07:58 (0)$ DAQR
	DAQRPC.py       DAQRunIface.py  DAQRun.py
	pdaq@sps-expcont[~] 05:07:58 (0)$ DAQR
	DAQRPC.py       DAQRunIface.py  DAQRun.py
	pdaq@sps-expcont[~] 05:07:58 (0)$ DAQRun.py -k
	pdaq@sps-expcont[~] 05:08:06 (0)$ !ps
	ps auxww | grep DAQ
	pdaq      8682  6.0  0.0 99476 7852 pts/2    Sl   05:02   0:20 python /usr/local/pdaq/pDAQ_current/dash/CnCServer.py -l localhost:9001
	pdaq     12532  0.0  0.0 51084  668 pts/2    S+   05:08   0:00 grep DAQ
	pdaq@sps-expcont[~] 05:08:07 (0)$ CnCServer.py -k
	pdaq@sps-expcont[~] 05:08:11 (0)$ !ps
	ps auxww | grep DAQ
	pdaq     12541  0.0  0.0 51084  668 pts/2    S+   05:08   0:00 grep DAQ


	pdaq@sps-expcont[~] 05:08:12 (0)$ DAQLaunch.py
	pdaq@sps-expcont[~] 05:08:51 (0)$ DAQStatus.py
	CNC localhost:8080
	-----------------------
	128 unused components
	idle
	  2 amandaTriggers: 0 0
	  3 eventBuilders: 0 0 0
	  3 globalTriggers: 0 0 0
	  3 iceTopTriggers: 0 0 0
	  3 inIceTriggers: 0 0 0
	  2 secondaryBuilderss: 0 0
	  112 stringHubs: 0 0 21 21 21 29 29 29-30 30 38 38 38-39 39-40 44 44 44-46
			  46-47 47 47-48 48 48-49 49 49-50 50 52 52-54 54 54-56 56-58
			  58-59 59 59-60 60-61 61 61-62 62-63 63 63-65 65-66 66-67
			  67-68 68-69 69 69-70 70 70-71 71 71-72 72-73 73 73-74 74
			  74-75 75-76 76 76-77 77 77-78 78 78 81 81 81-82 82 82-83 83
			  83-84 84 84-85 85-86 86 86
	-----------------------
	0 run sets
	DAQ state is STOPPED


0510Z	Another run attempt:
	pdaq@sps-expcont[~] 05:09:40 (0)$ ./starti3am
	Sub-system 'daq' are now under control of anvil.
	Sub-system 'twr' are now under control of anvil.
	2008-04-29 05:09:53: Started 'sam' service with PID=13581
	2008-04-29 05:09:53: Parameters for this run set:
	2008-04-29 05:09:53:           Run Mode: PhysicsTrig
	2008-04-29 05:09:53:          DAQ Label: sps-IC40-massive-icetop-changes-V043
	2008-04-29 05:09:53:          JeB Label: MergeEvents (1)
	2008-04-29 05:09:53:          PnF Label: PhysicsFiltering (1)
	2008-04-29 05:09:53:         Run Length: 28800
	2008-04-29 05:09:53:     Number of Runs: 40000
	2008-04-29 05:09:53: Started 'run' service
	2008-04-29 05:09:53: Parameters for this run:
	2008-04-29 05:09:53:         Run Number: 110911
	2008-04-29 05:09:53:           Run Mode: PhysicsTrig
	2008-04-29 05:09:53:          DAQ Label: sps-IC40-massive-icetop-changes-V043
	2008-04-29 05:09:53:          JeB Label: MergeEvents (1)
	2008-04-29 05:09:53:          PnF Label: PhysicsFiltering (1)
	2008-04-29 05:09:53: Starting run 110911
	2008-04-29 05:10:05: Starting 'twr' sub-system
	2008-04-29 05:10:05: Starting 'daq' sub-system
	2008-04-29 05:11:32: Started run 110911

	And events:
	DAQRun [2008-04-29 05:11:30.812100] Started run 110911 on run set 1
	DAQRun [2008-04-29 05:11:31.572477]     0 physics events (0.00 Hz), 28030 moni events, 4906 SN events, 11 tcals
	DAQRun [2008-04-29 05:12:01.304007]     50646 physics events (1533.00 Hz), 391908 moni events, 78246 SN events, 7930 tcals
	DAQRun [2008-04-29 05:12:31.581444]     90955 physics events (1436.56 Hz), 505681 moni events, 152226 SN events, 81034 tcals

	Put this info on skype and emailing run coordination.

	Downtime was:
	04:55:30 to 05:11:30 or 16 minutes.

Edgar Nielsen | 29 Apr 2008 00:18 GMT | Ice Cube/Runs | |