November 2009
Sun	Mon	Tue	Wed	Thu	Fri	Sat
1	2	3	4	5	6	7
8	9	10	11	12	13	14
15	16	17	18	19	20	21
22	23	24	25	26	27	28
29	30
Jun 20, 2008

Standard Candle 2 Run

By: TeX
Time: 20080619 2210Z
Music: Fiona Apple

2215Z	Inside an NX session on ic-cntrlws-02, copied
	the standard candle files to my home directory:
	cp -r ~dagostmi/standard_candle_uber_folder/standard_candle_2_code/ .

	Starting the prep work portion of the documentation.

2216Z	SC_client up, turning on the 3 heaters
2217Z	heaters on. Note starting temps:
	laser(external)	-26.6C
	laser(internal) 0C
	optics -25.9C
	electronics -25.2C

	Music now the B-52s.

2240Z	Working on getting screen dumps of the gui for each step
	to help the docs.


2328Z	Skype is up. Temps are now:
	laser(external)	-11.3C
	laser(internal) 0C
	optics -22.3C
	electronics -11.1C


2008172 0025Z	heaters off
	laser main on
	Oops, forgot about the run, laser main off
	emailing run coordination.
	heaters on

	pdaq@sps-expcont[~] 00:25:48 (0)$ anvil check
	Passed 'sn', in state 'Ignored'
	Passed 'twr', in state 'Started'
	Passed 'cluster', in state 'Ignored'
	Passed 'daq', in state 'Started'
	Passed 'spade', in state 'Ignored'
	Passed 'pnf', in state 'Started'
	pdaq@sps-expcont[~] 00:25:56 (0)$ anvil stop sam
	'sam' Stopped
	pdaq@sps-expcont[~] 00:28:07 (0)$


	sps-expcont[~] 00:28:45 (0)$ anvil control daq
	Sub-system 'daq' are now under control of anvil.
	pdaq@sps-expcont[~] 00:28:55 (0)$ anvil control twr
	Sub-system 'twr' are now under control of anvil.
	pdaq@sps-expcont[~] 00:28:59 (0)$ anvil start -j MergeEvents -p PhysicsFiltering -d sps-IC40-massive-icetop-changes-V043 -r TestData -n 40000 -l 28800 -v sam
	2008-06-20 00:30:57: Started 'sam' service with PID=21326
	2008-06-20 00:30:57: Parameters for this run set:
	2008-06-20 00:30:57:           Run Mode: TestData
	2008-06-20 00:30:57:          DAQ Label: sps-IC40-massive-icetop-changes-V043
	2008-06-20 00:30:57:          JeB Label: MergeEvents (1)
	2008-06-20 00:30:57:          PnF Label: PhysicsFiltering (1)
	2008-06-20 00:30:57:         Run Length: 28800
	2008-06-20 00:30:57:     Number of Runs: 40000
	2008-06-20 00:30:58: Started 'run' service
	2008-06-20 00:30:58: Parameters for this run:
	2008-06-20 00:30:58:         Run Number: 111186
	2008-06-20 00:30:58:           Run Mode: TestData
	2008-06-20 00:30:58:          DAQ Label: sps-IC40-massive-icetop-changes-V043
	2008-06-20 00:30:58:          JeB Label: MergeEvents (1)
	2008-06-20 00:30:58:          PnF Label: PhysicsFiltering (1)
	2008-06-20 00:30:58: Starting run 111186
	2008-06-20 00:31:09: Starting 'twr' sub-system
	2008-06-20 00:31:09: Starting 'daq' sub-system
	2008-06-20 00:32:11: Started run 111186


	The rest is from my skype log:

	[Thu Jun 19 2008 11:35:34] Tex: 2328Z   Skype is up. Temps are now:
		laser(external) -11.3C
		laser(internal) 0C
		optics -22.3C
		electronics -11.1C
	[Thu Jun 19 2008 11:37:00] Ignacio Taboada: Good. The optics are at the bottom of the device, the heat sources are in the middle and top. The optics have baffles that may stop air flow.
	[Thu Jun 19 2008 11:37:23] … I guess it's normal that optics are cold. It should be of no big concern.
	[Thu Jun 19 2008 11:37:31] * Ignacio Taboada invited michelangelo_dag
	[Thu Jun 19 2008 11:38:35] Tex: do you want to try for the -5C?
	[Thu Jun 19 2008 11:39:02] Ignacio Taboada: Let me guestimate how long we have to wait ...
	[Thu Jun 19 2008 11:39:22] Tex: we are now at -10.8C for laser ext
	[Thu Jun 19 2008 11:39:48] Michelangelo D'Agostino: it started at -25C?
	[Thu Jun 19 2008 11:39:57] Ignacio Taboada: Yes
	[Thu Jun 19 2008 11:40:10] Michelangelo D'Agostino: wow.  all that extra heating has really helped!
	[Thu Jun 19 2008 11:40:19] Tex: -10.4C now
	[Thu Jun 19 2008 11:40:39] Ignacio Taboada: So, it came down 15 C in 1 hour. We can afford to wait until it's at -5 C.
	[Thu Jun 19 2008 11:40:49] Tex: okily dokily
	[Thu Jun 19 2008 11:45:00] … -10C
	[Thu Jun 19 2008 11:53:23] Ignacio Taboada: I am going home soon. Let's say that we get started with the data taking at 00:15Z or at -5C, whichever is sooner.
	[Thu Jun 19 2008 11:53:43] … I'll skype in again once I'm at home.
	[Thu Jun 19 2008 11:53:46] Michelangelo D'Agostino: sounds good.
	[Thu Jun 19 2008 11:54:03] … tex, i can answer random questions while he's gone.
	[Thu Jun 19 2008 11:54:21] Tex: what is the airspeed of an unladen swallow?
	[Thu Jun 19 2008 11:54:45] Michelangelo D'Agostino: ignacio hasn't left yet.
	[Thu Jun 19 2008 11:55:41] Tex: I'm running downstairs to put the water  on for some delicious ramen
	[12:00:18] … -9.1C
	[12:03:48] … -8.6
	[12:12:09] Tex: -8.2C
	[12:13:06] Michelangelo D'Agostino: when were we supposed to stop?
	[12:13:17] … at -5C
	[12:13:26] … or 12:15
	[12:17:55] Tex: -7.8C
	[12:18:28] Michelangelo D'Agostino: tex--what directory did you install your code in?
	[12:18:33] … or i mean copy it to?
	[12:18:38] Tex: my home dir
	[12:18:45] Michelangelo D'Agostino: i need to have a look at the output files once we start running.
	[12:19:04] Tex: ! I'll open up my permissions then - wait a bit
	[12:19:14] Michelangelo D'Agostino: okay.
	[12:19:19] … so paranoid.
	[12:19:25] … :)
	[12:21:03] Tex: how's about now, you should be able to get there
	[12:21:57] … -7.8C
	[12:22:00] Mark: tex just opened up his directory?
	I am loggin g in to take a look...
	[12:22:13] Tex: don't worrry, I tightened up the perms on the subdirectories
	[12:22:48] Mark: hmm - i just managed to copy your ssh keys!
	[12:22:50] Tex: but all the good stuff is on t hecomputer in my room,  I work off of there, I'm typing on that machine now, I'm doing a remote desktop over NX
	[12:23:08] … !
	[12:24:40] … ok it is 12:18 so we should start, temp is -7.8C
	[12:25:45] Michelangelo D'Agostino: okay.  sounds good.  let's start then.
	[12:26:08] … according to the SOP, that is.
	[12:26:34] … so i guess that would be turn the heaters off, then turn on the laser main.
	[12:26:34] Tex: I'm going to turn off th eheaters, then the laser as per steps 10-12
	[12:26:35] Michelangelo D'Agostino: give it a second or two.
	[12:26:45] Tex: between heaters and laser?
	[12:26:46] Michelangelo D'Agostino: then the laser HV.
	[12:26:47] … yep.
	[12:26:52] … steps 10-12.
	[12:27:04] … nah.
	[12:27:22] … between the laser main and then the laser HV.
	[12:27:32] Tex: ok, turning off heaters now
	[12:29:09] … laser main on
	[12:29:22] … ooops the run!
	[12:29:30] … turned lasaer  minoff
	[12:29:40] … main off I mean,
	[12:29:49] Michelangelo D'Agostino: ah, right.
	[12:29:52] … we were just gonna spin the filter wheel a bit.
	[12:30:02] … but go ahead and start the new run first.
	[12:31:37] Tex: email sent to run coordination, now doing the stopping
	[12:32:14] Michelangelo D'Agostino: okay.
	[12:32:37] Tex: I put the heaters back on while I do this.
	[12:35:17] Michelangelo D'Agostino: sure.
	[12:35:50] Tex: this look good? anvil start -j MergeEvents -p PhysicsFiltering -d sps-IC40-massive-icetop-changes-V043 -r TestData -n 40000 -l 28800 -v sam
	[12:36:01] … the TestData is the only diff correct?
	[12:36:22] Michelangelo D'Agostino: you're asking the wrong person.  i'm not a daq guy.
	[12:36:34] … all i know is that it should be identical to a normal run, just tagged with TestData
	[12:37:01] Tex: that is the only thing ignacio said, so starting it up
	[12:37:29] Mark: any testdata runs in the expcont history?
	[12:37:43] Keith Beattie: I haven't been following this too closely but that looks correct Tex
	[12:38:17] … looking through the history, or in the various 'start' scripts is a good idea
	[12:38:34] Tex: that is what I did
	[12:38:38] Mark: looks right to me
	[12:38:55] Tex: run is started, ignacio said to let it run for a couple of minutes first
	[12:38:57] Dawn Williams: looks correct
	[12:39:41] Tex: events coming in
	[12:40:12] Michelangelo D'Agostino: cool.
	[12:40:17] … let's let it run for a few minutes.
	[12:40:28] … meanwhile, it's okay to turn the heaters off and the laser on.
	[12:40:42] Tex: okily dokily
	[12:41:26] … heaters off
	[12:41:39] … laser main on
	[12:41:51] Michelangelo D'Agostino: okay.
	[12:41:55] … give it a second or two.
	[12:41:59] … then turn the HV on.
	[12:42:10] Tex: laser hv on
	[12:42:18] Michelangelo D'Agostino: after the HV, would you run and check the current on the genesys?
	[12:42:49] Tex: .077
	[12:43:03] … temp is -7.3C
	[12:43:52] Michelangelo D'Agostino: great.  looks good.
	[12:44:12] … so i guess now we want to check out the filter wheel.
	[12:44:23] … so ignacio said he should have added to this to the SOP.
	[12:44:31] … we're going to set the filter wheel to 100%.
	[12:44:37] … and then down to .5%
	[12:44:40] … and then back to 100%
	[12:44:48] Tex: what does this do?
	[12:45:08] Michelangelo D'Agostino: just sort of wakes it up.
	[12:45:15] … when it's cold, the filter wheel can stick.
	[12:45:23] … so we wanna try to get it moving.
	[12:45:30] … so give it a try.
	[12:45:41] Tex: so trigger:burst, filter 100% what about rate and pulses settings for this? can they be empty
	[12:46:08] Michelangelo D'Agostino: select burst, because that's what we're going to use.
	[12:46:13] … you can leave the other ones empty for now.
	[12:46:22] … just type 100 in the filter wheel box and hit return.
	[12:46:30] … you should then see the value next to the % start to climb up.
	[12:47:04] Tex: started
	[12:47:21] … done I guess, doing .5% now
	[12:47:23] Michelangelo D'Agostino: it's moving up to 100?
	[12:47:35] … awesome!
	[12:47:37] Tex: it went to 10
	[12:47:40] … it went to 100
	[12:47:45] … now is .5%
	[12:47:53] … should I do an .5% explicitly?
	[12:48:11] … wait, laser hv is now off!
	[12:48:30] Michelangelo D'Agostino: huh.
	[12:48:35] Tex: does it that automatically after a run now?
	[12:48:37] Michelangelo D'Agostino: is the laser main still on?
	[12:48:53] … wait, did you hit start or stop run?
	[12:48:56] Tex: yes
	[12:49:06] … I hit return like you said to
	[12:49:08] Michelangelo D'Agostino: ah, okay.
	[12:49:15] … but did you click the button?
	[12:49:21] … the start run button?
	[12:49:23] Tex: nope
	[12:49:37] Michelangelo D'Agostino: so you just typed a number into the filter wheel box and hit return?
	[12:49:42] Tex: exactly
	[12:49:47] Michelangelo D'Agostino: ok.
	[12:49:56] … well, i think we should power cycle if it did that.
	[12:50:16] … can you turn off the laser main?
	[12:50:22] Tex: ok, so I'll turn off laser main, then back on then laser hv on?
	[12:50:31] Michelangelo D'Agostino: let's try that.
	[12:50:40] Tex: laser main cycled
	[12:50:54] … laser hv on
	[12:51:04] … should we try the wheel again?
	[12:51:36] Michelangelo D'Agostino: sure.  set it to 100, and then hit return.
	[12:51:46] Tex: but no click on a button?
	[12:51:57] Michelangelo D'Agostino: no.
	[12:52:05] … just hitting enter should send the value to the laser.
	[12:52:22] Tex: done, went to 100, laser hv is on
	[12:52:34] Michelangelo D'Agostino: okay.
	[12:52:47] … now type 10 and hit return.
	[12:53:21] Tex: went to 10, laser hv still on
	[12:54:08] Michelangelo D'Agostino: okay.  good.
	[12:54:16] … now type 100 and hit return.
	[12:54:25] … we're gonna do fifty pulses or so and i'll look at the log file to make sure it's pulsing properly.
	[12:54:34] Tex: went to 100, laser hv still on
	[12:55:00] … so I should put pulses at 50 and all else empty?
	[12:55:13] Michelangelo D'Agostino: make sure burst is selected.
	[12:55:21] Tex: do I just hit return still?
	[12:55:22] Michelangelo D'Agostino: type 50 for pulses and hit return
	[12:55:27] … yep.
	[12:55:34] Tex: done
	[12:55:42] Michelangelo D'Agostino: (don't click start until we actually want the laser to pulse)
	[12:55:55] … then type 1 Hz into the rate box and hit return
	[12:56:00] Tex: what is pulsing now then?
	[12:56:12] … so 50 pulses and 1Hz or just 1Hz by itself?
	[12:56:24] Michelangelo D'Agostino: nothing is actually pulsing now.
	[12:56:33] … we're just rotating the wheel and setting things up.
	[12:56:47] Tex: do I type 1, 1Hz, or 1 Hz, does it care?
	[12:56:47] Michelangelo D'Agostino: the laser doesn't pulse until we click "start run"
	[12:56:56] … just type 1 and hit return
	[12:57:06] … i'll be more explicity.  sorry.
	[12:57:12] Tex: done
	[12:57:16] Michelangelo D'Agostino: we probably didn't put in all the right error checking.
	[12:57:41] … okay.  so just to confirm we have "burst mode", "1 Hz", "50 pulses", and "100%"
	[12:57:51] Tex: oh all at once?
	[12:58:24] Michelangelo D'Agostino: you can type each one in separately and then it should display the values in the field just to the right of each box.  see what i mean?
	[12:58:38] Tex: once I hit return you mean
	[12:58:45] Michelangelo D'Agostino: yep.
	[12:58:53] Tex: done now
	[12:58:54] Michelangelo D'Agostino: so type each value in its box, hit return, and the value should appear to the right.
	[12:58:57] … okay.
	[12:59:21] … so then we're ready to start.  click "start run", and you should see some data scrolling down the message field.
	[12:59:25] … i'll look at the .dat file.
	[12:59:31] Tex: here it goes
	[12:59:43] … running
	[13:01:30] Michelangelo D'Agostino: so did the output stop?
	[13:01:40] Tex: seems to have stopped at 61
	[13:01:49] Michelangelo D'Agostino: yeah, that's what i see.
	[13:01:58] Tex: the logfile has: 2008-06-20 12:53:28.449- received: command acknowledged
	2008-06-20 12:53:28.468- received: ERR: did not understand laser communication
	[13:02:05] Michelangelo D'Agostino: yeah, i just saw that.
	[13:02:50] … click "stop run"
	[13:02:51] Tex: laser hv is now off
	[13:03:06] Michelangelo D'Agostino: shit.
	[13:03:14] … just saw this intermittent behavior right before we deployed.
	[13:03:22] … are you in the ICL?
	[13:03:28] Tex: but of course
	[13:03:36] Michelangelo D'Agostino: okay.  good.
	[13:03:50] … i think that the best thing to do is to power cycle on the actual genesys.
	[13:04:00] … so make sure that the hv is off.
	[13:04:02] … then turn the laser main off.
	[13:04:31] Tex: ok, turning off last main, the power cycling the gensys. Should I exit the gui?
	[13:04:54] Michelangelo D'Agostino: yeah, turn of the laser main and quit the GUI.
	[13:05:04] … and then power cycle the genesys.
	[13:05:18] Tex: hv main off, exiting the gui
	[13:05:35] … ok, powercycling the genesys
	[13:08:35] … when I turned it off, for a bit it had the alarm led and said AC
	[13:08:58] … after turning all back on is .037A instead of .039A is that ok?
	[13:09:15] Michelangelo D'Agostino: that's okay.
	[13:09:22] Tex: gui is up
	[13:09:36] Michelangelo D'Agostino: the alarm is not a huge deal (yet)
	[13:09:48] … so now the gui is back up.
	[13:09:53] Tex: note that I did nont uplug the DSL extender during this
	[13:09:58] Michelangelo D'Agostino: turn the laser main on and then the laser hv.
	[13:10:09] … (that's okay not to cycle the extender...)
	[13:10:43] Tex: laser main on
	[13:11:02] … laser hv on (laser ext temp is -13.0C)
	[13:11:46] Michelangelo D'Agostino: okay.
	[13:11:45] … hold on a second.  i got booted off the satty.
	[13:12:58] … okay.   back.
	[13:12:59] Tex: must be going to TDRSS
	[13:13:03] Michelangelo D'Agostino: so select "burst"
	[13:13:18] Tex: yep
	[13:13:51] Michelangelo D'Agostino: now type "1" in the trigger rate box and hit enter
	[13:14:03] Tex: done
	[13:14:12] … laser hv still on
	[13:15:02] Michelangelo D'Agostino: now type "50" in pulses and hit enter.
	[13:15:15] Tex: ok
	[13:16:11] Michelangelo D'Agostino: okay.  no type "100" in the filter wheel box and hit enter.
	[13:16:28] Tex: wait, laser hv is off!
	[13:17:22] … huh.
	[13:17:23] … okay.
	[13:17:48] Tex: I did the laser power cycle, laser hv back on
	[13:18:09] Michelangelo D'Agostino: okay.  let's keep going.
	[13:18:21] … just remembers this happening intermittently (and annoyingly....sorry...)
	[13:18:24] Tex: should  Ido the 100 now
	[13:18:35] Michelangelo D'Agostino: he said that a combination of power cycling and quitting and restarting the gui helped.
	[13:18:46] … yes.  try it again.
	[13:18:51] Tex: so it is 1Hz, 50 pulses and FW 100
	[13:19:07] Michelangelo D'Agostino: yes.
	[13:19:19] Tex: done, went to 100, laser hv on
	[13:19:45] Michelangelo D'Agostino: okay.  so let's click "start run" again.
	[13:19:54] Tex: here it goes
	[13:20:03] … started, laser hv on
	[13:21:16] Michelangelo D'Agostino: good.  it seems to be working.
	[13:21:31] Tex: stopped at 61 laser hv still on
	[13:21:34] Michelangelo D'Agostino: the power values coming out in the data file look okay.
	[13:21:44] Tex: -14.8C
	[13:22:26] Michelangelo D'Agostino: okay.  i think we're ready to start taking data.
	[13:22:57] Tex: do I need to turn laser off and restart gui?
	[13:23:17] Ignacio Taboada: Hi again
	[13:23:31] Michelangelo D'Agostino: yeah, sorry.  so turn off the HV and then the laser main, hit quit, and reopen the gui.
	[13:23:36] … and cross your fingers :)
	[13:23:38] … hey ignacio.
	[13:23:45] … we were having that sort of intermitten HV problem.
	[13:24:08] Tex: done, starting up gui
	[13:24:28] … laser main on, laser hv on
	[13:24:32] Michelangelo D'Agostino: good.
	[13:25:06] … now select "burst" and enter "2000" for pulses and "1" for rate.
	[13:25:33] Tex: done, FW is still .5%
	[13:26:10] Michelangelo D'Agostino: did you type in ".5" and hit return for the FW?  do that again just to be safe.
	[13:26:15] Tex: is this gui going to be incorporated into Icecube Live?:O
	[13:26:28] … I did it now
	[13:26:30] Michelangelo D'Agostino: that's after my time :)
	[13:26:34] … okay.
	[13:26:38] … so all the values are set?
	[13:26:48] Tex: 1Hz, 2000 pulses and FW .5%
	[13:26:57] … laser hv still on
	[13:27:02] Michelangelo D'Agostino: great!
	[13:27:07] … so we're ready to go.
	[13:27:14] … click "start run" and we're off.
	[13:27:19] Tex: here it goes
	[13:27:27] … started, laser hv still on
	[13:27:53] … anyone see 'the happening'?  is it as bad as the reviews say?
	[13:28:41] Michelangelo D'Agostino: hey ignacio.   the power meter readings are still coming out like "10: 8917 111.88"
	[13:28:49] … does that mean the filter wheel is at 100% and not .5%?
	[13:28:57] Ignacio Taboada: No.
	[13:29:01] Tex: it says .5%, still runnnig
	[13:29:05] Ignacio Taboada: The power measurement is before the filter wheel
	[13:29:28] Michelangelo D'Agostino: ah right.  good.
	[13:29:46] Tex: is the leftmost number in the output window going to hit 2000?
	[13:29:45] Ignacio Taboada: I read about the HV issues you had - could it be the laser BIOS going to sleep?
	[13:29:51] Michelangelo D'Agostino: so that last number shouldn't change according to the filter wheel setting?
	[13:30:03] … tex, it should.
	[13:30:04] Ignacio Taboada: Michelangelo - right
	[13:30:04] Michelangelo D'Agostino: and a little bit beyond...
	[13:31:21] Tex: ok, at 230 now, laser hv still on, -16.1C
	[13:31:48] Michelangelo D'Agostino: all good, for now.
	[13:32:04] Ignacio Taboada: We may need to put the heaters on between the settings if we find that the laser gets cold, but -16 C is good.
	[13:32:25] Tex: I'm not sure why ya'll don't run this yourselves with an NX remote desktop. I run remote desktops on my LA server and  it works pretty well. www.nomachine.com
	[13:32:28] Ignacio Taboada: Michelangelo, did you check the log to see the response to the commands?
	[13:33:09] … Tex: Michelangelo had begun writing an ncurses interface but that work had to stop - long story
	[13:33:23] Michelangelo D'Agostino: i did check the log.
	[13:33:26] … things looked okay.
	[13:33:28] Ignacio Taboada: I agree it'd be nicer if we could run ourselves.
	[13:33:31] Michelangelo D'Agostino: there was an error or two.
	[13:33:36] Ignacio Taboada: i.e.?
	[13:34:21] Michelangelo D'Agostino: errors in the response from the laser, i think.  it didn't properly acknowledge a received command or two.
	[13:34:57] Ignacio Taboada: Hmm. Could it be a bug in the client?
	[13:36:25] Michelangelo D'Agostino: i dunno.  hard to tell.
	[13:38:06] Ignacio Taboada: What is the data file you're looking at, Michelangelo?
	[13:38:48] Michelangelo D'Agostino: SC_06202008_13\:17\:53 .dat and .log in /home/enielsen/standard_candle_2_code
	[13:39:43] Ignacio Taboada: OK. I see the pulses going.
	[13:39:51] … I'll be back in ~20 minutes
	[13:39:59] Michelangelo D'Agostino: okay.
	[13:41:12] Tex: 815, -16.5C, laser hv still on
	[13:41:44] Michelangelo D'Agostino: good!
	[13:41:51] … we may need to turn the heaters on for a bit after this run.
	[13:42:06] … (btw, i'm watching the laser output too...)
	[13:42:48] Tex: I figured, but I'm going to edit this down for my logbook entry! And I want to make sure I keep an eye on that laser hv status
	[13:43:05] Michelangelo D'Agostino: good idea!
	[13:52:04] Tex: 1470, -16.5, laser hv on
	[14:00:59] Michelangelo D'Agostino: almost there...
	[14:01:18] Tex: 2011, -16.5C, laser  hvon, seems to have stopped now
	[14:01:39] Michelangelo D'Agostino: yeah, it's done.
	[14:01:49] Tex: should I turn off the lasers and turn on the heaters?
	[14:02:10] Michelangelo D'Agostino: let's do that.  hv off first, then main.
	[14:02:16] … we'll let it heat up for a bit.
	[14:02:29] Tex: lasers off, heaters on
	[14:04:19] Michelangelo D'Agostino: good.
	[14:04:44] … so midwinter is coming up, huh?
	[14:05:09] Tex: dinner is tonite, and a 2 day weekend
	[14:05:32] Ignacio Taboada: GOOOOOOOOOOL (party)
	[14:05:36] Michelangelo D'Agostino: nice!  a 2 day weekend.  the stuff they make you guys put up with down there...
	[14:05:39] Ignacio Taboada: Venezuela 1 Chile 0 ....
	[14:05:44] … I'm back by the way
	[14:05:48] Michelangelo D'Agostino: so that's why i had to take this shift :)
	[14:06:01] … we're just running the heaters again.
	[14:06:20] Ignacio Taboada: Unfortunately I'm not watching. I'm reading a blog. Yes, I read the notes
	[14:07:21] … So, the laser seems to maintain temperature while running.
	[14:07:44] Tex: -16.1C now
	[14:07:49] Ignacio Taboada: I guess that we can just let the temp go up a bit, say to -14 C and start with the next setting.
	[14:08:36] Michelangelo D'Agostino: okay.
	[14:13:01] … -15.2C
	[14:15:46] … -14.3C
	[14:16:01] Ignacio Taboada: Good enough for gov' work
	[14:16:12] Tex: so I should stop the heaters and restart gui?
	[14:16:17] Ignacio Taboada: Yeap
	[14:16:38] Tex: ok, gui back up now
	[14:16:40] Michelangelo D'Agostino: hey ignacio.
	[14:16:52] … i'll wait until this run is up and going again and then i'm gonna head off.
	[14:16:58] Ignacio Taboada: OK
	[14:17:08] Tex: laser main and laser hv on
	[14:17:28] Michelangelo D'Agostino: check this plot out: http://icecube.berkeley.edu/~mdagost/atmcascades/double_deuce_graphs_postMadison/reducedLlhComparisonLevel4.gif
	[14:17:41] … it's at level 4 already, so i can still clamp down on that a bit.
	[14:18:23] Tex: trigger burst, trigger rate 1 Hz, Pulses 2000, FW .5% are current settings
	[14:18:24] Michelangelo D'Agostino: tex-are you selecting "burst" and seting the rate to "1"?
	[14:18:29] … cool.
	[14:18:45] … type "1" in the FW box and hit return.
	[14:18:58] Tex: done, 1.0%
	[14:19:41] … should I start the run? laser hv still is on
	[14:19:48] Michelangelo D'Agostino: yep!  go for it.
	[14:19:58] Tex: run is started
	[14:20:03] … -13C
	[14:20:32] Ignacio Taboada: I saw it MVD. Looks good - check ndirc too.
	[14:21:29] Michelangelo D'Agostino: yeah, i will.  i think the timing offset is understood too.  we'll talk tomorrow.
	[14:21:39] … tex, the numbers in the datastream look good.
	[14:25:13] Tex: cool, 315, -13.0C laser hv still on
	[14:35:38] Tex: 935, -14.3C, laser hv on
	[14:36:12] Ignacio Taboada: Ahh, so the laser does cool down slowly while running. Good to know.
	[14:36:26] Michelangelo D'Agostino: hey ignacio--did you see those emails? looks like the PMT transit time is the reason for the ~100 ns timing offset.
	[14:36:51] Ignacio Taboada: I thought that PMT transit is less than that - I could be wrong. Yes, I saw the e-mails
	[14:37:35] Michelangelo D'Agostino: it's hardcoded into the FE as around 141ns plus some HV dependent factor.
	[14:37:42] … it's the PMT transit and the delay line together.
	[14:37:46] Ignacio Taboada: Ahh ....
	[14:43:51] … 1430, -14.8, laser hv on
	[14:44:37] Michelangelo D'Agostino: okay guys.  i think i'm gonna take off.
	[14:44:47] … is that cool?
	[14:44:58] Ignacio Taboada: Cool.
	[14:45:03] Tex: sure
	[14:45:08] Michelangelo D'Agostino: okay.
	[14:45:22] … ignacio, if the hv problem acts up again, trying turning things off and restarting the gui.
	[14:45:33] … and if that  doesn't work, then shut down the gui and power cycle the genesys.
	[14:45:42] … this was what justin remembered doing too.
	[14:46:07] Ignacio Taboada: OK.
	[14:46:13] Michelangelo D'Agostino: have fun!
	[14:49:34] Tex: 1770, -15.2C, laser hv on
	[14:53:54] … 2011, -15.2C, laser hv on, run finished?
	[14:53:57] Ignacio Taboada: OK. Almost done. Same as before please turn on the heaters until we are close to -14C
	[14:54:18] Tex: laser hv off, laser main off, heaters on
	[14:59:59] … -14.8C
	[15:02:27] … -14.3C
	[15:02:45] Ignacio Taboada: OK. You can go ahead with the next setting
	[15:02:59] Tex: turning off heaters and restarting gui first
	[15:03:36] … gui restarted, laser main on , laser hv on
	[15:03:48] Ignacio Taboada: Next setting is filter wheel 3
	[15:03:56] … burst mode, 2000  pulses, 1 Hz
	[15:04:04] Tex: ready to go
	[15:04:11] Ignacio Taboada: Fire at will
	[15:04:21] Tex: started, -13.5C, laser hv on
	[15:05:03] Ignacio Taboada: Good. I see the pulses
	[15:11:47] Tex: 450, -13.5C, laser hv on
	[15:17:54] Azriel Goldschmidt: Hey guys! how's going?
	[15:18:13] Ignacio Taboada: Going well. No significant issues with the SC2
	[15:18:19] Azriel Goldschmidt: I forgot to think and talk about data disposition for the SCandle2 run
	[15:18:24] Ignacio Taboada: It'll be a long night for me ...
	[15:18:30] … I've thought about it.
	[15:18:46] Azriel Goldschmidt: will you grab it with grb style?
	[15:19:00] Ignacio Taboada: The data can be taken from the PnF master disk (by hand - I'd rather not use GRB filter).
	[15:19:09] … then we process it like last year
	[15:19:19] … and we only send north the SC events.
	[15:19:19] Tex: azriel, did you get to the tbdaq machines?
	[15:19:25] Ignacio Taboada: So the total data volume is small.
	[15:19:38] Tex: 920, -13.9C, laser hv still on
	[15:19:48] Azriel Goldschmidt: that is ok, so you will do that work before the data dissapears in a ~days
	[15:20:02] … ?
	[15:20:16] Ignacio Taboada: I'll move the data today.
	[15:21:26] Azriel Goldschmidt: super, thanks for thinking ahead. I will check now with Mark about the possibility of making a special config for atwdch2 issue...
	[15:22:05] Ignacio Taboada: Tex, what is the current pDAQ run number?
	[15:23:11] Tex: hmm it is:
	[15:23:21] … 111186
	[15:23:35] Ignacio Taboada: thx
	[15:23:46] Tex: laser internal temp 21C?
	[15:23:55] Ignacio Taboada: hmm?
	[15:24:03] Tex: that is what it says now
	[15:24:06] Ignacio Taboada: Wierd .
	[15:24:20] … What are the other temps?
	[15:24:29] Tex: laser ext -14.3C
	[15:24:36] … optics -21.9C
	[15:24:44] … electronics -11.1C
	[15:25:21] Ignacio Taboada: It's hard to imagine a >20 C temperature differential. Hmm. Let's take a pause after this run.
	[15:25:39] … Please turn on only one of the 12 W heaters and let's see what happens.
	[15:25:53] Tex: ok
	[15:25:55] Ignacio Taboada: turn on the heater once we are done with this run, of course
	[15:26:00] Tex: yep
	[15:30:50] … 1580, -14.8C, laser hv on
	[15:37:20] … 1970, -15.2C, laser hv on
	[15:38:44] Ignacio Taboada: OK. Run over.
	[15:39:16] Tex: 2011, -15.2C, laser hv on, run finishe
	[15:39:36] … lasers off, heater #2 only on
	[15:39:44] … #3 I mean
	[15:39:50] Ignacio Taboada: Cool, now we wait
	[15:40:20] Tex: what about turning off the heaters, restarting the gui to see  if the 21C is a bug or not?
	[15:42:35] Ignacio Taboada: Yes sure. Go for it.
	[15:43:06] Tex: still says 21C, heater #3 is on
	[15:43:53] Ignacio Taboada: OK. Let's keep an eye on the other temperatures.
	[15:44:10] Tex: all are about the same still
	[15:46:01] Ignacio Taboada: Let's wait ~10 minutes and get started again.
	[15:46:16] Tex: ok, temp still stuck at -15.4C
	[15:46:17] Ignacio Taboada: My hope is that temperature will uniformize within the SC2
	[15:46:28] Tex: uniformize - is that a real word?
	[15:50:58] … laser external -15.4C
	laser internal 21C
	optics -22.3C
	electronics -11.7C
	[15:51:45] Ignacio Taboada: I have now clue about uniformize - I probably just made it up :-)
	[15:52:24] Tex: temps not changing, 1 htr not enough it seems
	[15:53:37] Ignacio Taboada: I think we wasnt enough heat to maintain average termp
	[15:54:06] … while letting the laser (internal) to cool down
	[15:54:25] … * i meant want *
	[15:55:18] Tex: are the positions of the heaters different, ie one farther away from the internal portion of the laser?
	[15:55:47] Ignacio Taboada: No. All three heaters are wrapped around the laser.
	[15:55:59] … The temp should be high to help the stepper motor.
	[15:56:16] … The temp should be below freezing to eliminate the risk of condensation
	[15:56:45] Tex: or so high to evaporate it? Like 100C?
	[15:56:46] Ignacio Taboada: We let the SC aerate in dry south pole air before deploying so condensation risk is small
	[15:57:05] Tex: ooh what about a dry argon gas bath?
	[15:57:29] Ignacio Taboada: We thought about using nitrogen (cheap) too late for that anyway
	[15:57:39] Tex: temperatures are still exactly the same!
	[15:58:03] Ignacio Taboada: Hmm. Ok, we are not progressing. So let's continue with the original plan.
	[15:58:24] Tex: ok, heater off, turning on lasers
	[15:58:38] Ignacio Taboada: Let's keep an eye in laser(internal). We may want to take more dramatic action if laser(internal) begins to go up.
	[15:58:46] Tex: laser main on, laser hv on
	[15:58:49] Ignacio Taboada: Not clear yet, whether that reading is real, though.
	[15:58:53] … Next setting:
	[15:59:20] … 10 %, burst mode, 2000 pulses, 1 Hz
	[15:59:29] Tex: done
	[15:59:39] Ignacio Taboada: fire at will
	[15:59:47] Tex: run started
	[16:00:30] Ignacio Taboada: Looking good
	[16:05:53] Tex: 370,-15.4C, laser hv on
	[16:06:07] … all temps the same but electronics is now -12.2C
	[16:06:23] Ignacio Taboada: OK
	[16:15:34] Tex: 945, -15.9C, laser hv on
	[16:19:56] … 1210,-16.3C,laser hv on
	[16:27:17] Ignacio Taboada: I have to leave for ~20 minutes. Please go on with pla. Next setting is
	30%, burst mode, 1 Hz, 2000 pulses.
	[16:27:37] Tex: ok
	[16:32:53] … 1980, -16.3C, laser hv on
	[16:34:11] … 2011, -16.3C, laser hv on
	[16:34:16] … run finished, lasers off
	[16:34:51] … since so cold, turning on 24W and 1 12W
	[16:38:20] … temp not moving, turned on 3rd heater.
	[16:43:07] … laser external is -15.4C, electronics -11.7C all others same
	[16:43:15] Ignacio Taboada: OK, I am back.
	[16:44:43] Tex: turned off heaters, restarted gui
	[16:44:50] … temp  is-15.2C, starting next run
	[16:44:59] … laser main on
	[16:45:11] … laser hv on
	[16:45:51] … fw says 30.5% instead of 30%, is that ok?
	[16:45:58] Ignacio Taboada: That's ok
	[16:46:13] Tex: run started
	[16:46:17] Ignacio Taboada: It's a digitization issue in the laser BIOS
	[16:46:31] Tex: lasere ext temp is -14.3C
	[16:46:48] Ignacio Taboada: looks good
	[16:46:54] Tex: that internal laser temp is stuck at 21C
	[16:47:20] Ignacio Taboada: I'll have to look into that. My guess is that it is a false reading
	[16:54:07] Tex: 480, -14.3C, laser hv on
	[17:01:11] … 900,-14.8C,laser hv on
	[17:06:31] … 1220,-15.2C,laser hv on
	[17:19:50] … 2011,-15.7C,laser hv on, run finished
	[17:19:57] … lasers off, heaters on
	[17:20:05] Ignacio Taboada: Cool. Now for 50%, 1 Hz, burst, 2000 pulses
	[17:20:24] Tex: should I warm up the laser external?
	[17:20:40] Ignacio Taboada: Yes, to -14 C as usual.
	[17:20:48] Tex: ok, all 3 heaters on
	[17:21:22] Azriel Goldschmidt: are the pdaq run/s going without incident? any way of knowing there is actual light in there?
	[17:21:58] Tex: not that I know of. the same run is still going 1272-1283Hz
	[17:22:23] Ignacio Taboada: Rates will not change significantly, since the SC is at 1 Hz.
	[17:22:44] Azriel Goldschmidt: do you know where the .moni log files are from pdaq during the run in action?
	[17:22:46] Ignacio Taboada: But the laser is flashing. I can check the data comming from the laser
	[17:23:16] Azriel Goldschmidt: ok, sounds like you are confident, so I will not bother to check hit rate increase
	[17:23:40] Ignacio Taboada: Is it easy? It could be interesting anyway.
	[17:23:55] Azriel Goldschmidt: easy, I think
	[17:27:17] Tex: -14.8C, restarting the GUI
	[17:27:53] … laser main on, laser hv on
	[17:28:19] … 1Hz, 2000 pulses, 51.0% fw
	[17:28:27] Ignacio Taboada: OK.
	[17:28:42] Tex: run started, -14.3C
	[17:28:56] … I'm going to sweep the snow off the decks for a bit....
	[17:29:16] Ignacio Taboada: Cool. I'll keep an eye on the log/data
	[17:36:38] Azriel Goldschmidt: remind me the sc2 location, I am looking at the moni log file
	[17:36:54] Ignacio Taboada: String 55 between DOMs 42 & 43
	[17:37:21] Azriel Goldschmidt: we are flashing now?
	[17:37:37] Ignacio Taboada: Yes, at 50%, so it should be bright
	[17:38:19] … I've begun copying the data to temporary disk, by the way.
	[17:47:48] Tex: 1150, -14.8C, laser hv still on
	[17:52:49] Azriel Goldschmidt: Typical hit rate for 55-42 is 8 Hz, now you are getting 24 Hz. I guess this is reasonable given all the late light and after pulses, etc. Definitely light out there
	[17:53:04] Ignacio Taboada: Cool
	[17:53:25] … Do we know if AMANDA is ok?
	[17:53:41] … (it's far and higher, so it may not be affected actually)
	[17:58:05] Azriel Goldschmidt: I do not have a good check for that, maybe Tex does... Trigger rate is currently 230Hz for AMANDA, is that reasonable?
	[17:58:54] Ignacio Taboada: I think yes - given the newish trigger for AMANDA
	[18:00:01] Tex: heck if I know!
	[18:00:37] Ignacio Taboada: I checked. 220 - 200 is the normal range since the deployment of the new trigger
	[18:00:47] … http://i3moni.icecube.wisc.edu/cgi-bin/2008/historical.pl?histo=TriggerType2DDay_AMANDA_TWR_DAQ_STRING&inputfile=/data/exp/IceCube/2008/monitoring/ProcNorth/combined.root
	[18:00:48] … :-)
	[18:01:45] Azriel Goldschmidt: ok, I am going to bed... good luck with the last stretch and making sure the candle is off at the end ! :) I will deal tomorrow with Mina's request for a short data taking with SCandle2 in the all channels mode (regardless of the issues with bandwidth, etc) It requires new configurations and a willing winterover to turn the candle again...
	[18:02:02] Tex: ours is not to reason why....
	[18:02:25] Ignacio Taboada: Thanks Tex. We know.
	[18:02:32] Azriel Goldschmidt: indeed
	[18:02:42] Tex: 2011, -15.2C, laser hv on, run finished
	[18:02:51] Ignacio Taboada: OK. Last one.
	[18:03:00] Tex: all 3 heaters on
	[18:03:02] Ignacio Taboada: 100%, 2000 pulses, burst, 1 Hz
	[18:05:04] … Last year the SC was left on by accident. And it went into backup mode. So we got a few free pulses :-)
	[18:08:50] … -14.8C,  reresttatinIgG hU
	[18:09:22] … GUI restarted, laser main on
	[18:09:34] … laser hv on
	[18:10:01] Tex: 1Hz, 2000 pulses, 100.0% fw
	[18:10:26] Tex: run started, -14.3C
	[18:11:03] Ignacio Taboada: data looks good.
	[18:15:01] … 280,-13.9C, laser hv on
	[18:15:24] … might as well do  some more shoveling
	[18:15:33] Ignacio Taboada: OK
	[18:30:07] Tex: 1180,  -14.8C,laser  hvon
	[18:44:09] … ok 2011, -15.2C, laser hv on we is done
	[18:44:40] … starting the turn off procedures
	[18:44:41] Ignacio Taboada: Done!
	[18:46:03] Tex: powered all off including the 3 DSL extenders, exiting gui
	[18:46:13] … that is all correct?
	[18:46:20] Ignacio Taboada: Excellent.
	[18:46:29] … Yes, that's it
	[18:46:43] … I'll make an entry in the changelog
	[18:46:52] Tex: okey dokey, I'm off to MAPO then the midwinter dinner. oops gotta do the new run too
	[18:46:55] Ignacio Taboada: Can you start the new pDAQ run and e-mail run-coordination?
	[18:47:00] Tex: yep
	[18:47:13] … stopping the run


Back in the real world:
	pdaq@sps-expcont[~] 06:40:53 (130)$ anvil stop sam
	'sam' Stopped
	pdaq@sps-expcont[~] 06:43:01 (0)$
	pdaq@sps-expcont[~] 06:43:09 (0)$ ./starti3am
	Sub-system 'daq' are now under control of anvil.
	Sub-system 'twr' are now under control of anvil.
	2008-06-20 06:43:13: Started 'sam' service with PID=17380
	2008-06-20 06:43:13: Parameters for this run set:
	2008-06-20 06:43:13:           Run Mode: PhysicsTrig
	2008-06-20 06:43:13:          DAQ Label: sps-IC40-massive-icetop-changes-V043
	2008-06-20 06:43:13:          JeB Label: MergeEvents (1)
	2008-06-20 06:43:13:          PnF Label: PhysicsFiltering (1)
	2008-06-20 06:43:13:         Run Length: 28800
	2008-06-20 06:43:13:     Number of Runs: 40000
	2008-06-20 06:43:13: Started 'run' service
	2008-06-20 06:43:13: Parameters for this run:
	2008-06-20 06:43:13:         Run Number: 111187
	2008-06-20 06:43:13:           Run Mode: PhysicsTrig
	2008-06-20 06:43:13:          DAQ Label: sps-IC40-massive-icetop-changes-V043
	2008-06-20 06:43:13:          JeB Label: MergeEvents (1)
	2008-06-20 06:43:13:          PnF Label: PhysicsFiltering (1)
	2008-06-20 06:43:13: Starting run 111187
	2008-06-20 06:43:25: Starting 'twr' sub-system
	2008-06-20 06:43:25: Starting 'daq' sub-system
	2008-06-20 06:44:31: Started run 111187

	Events coming in:
	DAQRun [2008-06-20 06:44:31.350303] Started run 111187 on run set 1
	DAQRun [2008-06-20 06:44:31.861712]     0 physics events (0.00 Hz), 35307 moni events, 2746 SN events, 4 tcals
	DAQRun [2008-06-20 06:45:01.831360]     44642 physics events (1357.89 Hz), 393494 moni events, 78231 SN events, 7026 tcals
	DAQRun [2008-06-20 06:45:32.081013]     83559 physics events (1323.69 Hz), 507158 moni events, 151544 SN events, 80672 tcals
	DAQRun [2008-06-20 06:46:02.292627]     123194 physics events (1319.88 Hz), 623613 moni events, 226657 SN events, 155556 tcals

	Emailing run coordination.
Edgar Nielsen | 20 Jun 2008 01:46 GMT | Ice Cube/Standard Candle | |
Sections

Search

Archives

Jun 20, 2008

Standard Candle 2 Run

November 2009
Sun	Mon	Tue	Wed	Thu	Fri	Sat
1	2	3	4	5	6	7
8	9	10	11	12	13	14
15	16	17	18	19	20	21
22	23	24	25	26	27	28
29	30
November 2009
Sun	Mon	Tue	Wed	Thu	Fri	Sat
1	2	3	4	5	6	7
8	9	10	11	12	13	14
15	16	17	18	19	20	21
22	23	24	25	26	27	28
29	30