Home
South Pole Logbook

Search below for 'logbook_sop' for help on usage.

Sections

Search

Archives

November 2009
Sun Mon Tue Wed Thu Fri Sat
         

RSS Feed

Powered by Blosxom


Nov 30, 2007

Found an AMANDA cable on the floor

See more ...


Edgar Nielsen | 30 Nov 2007 21:52 GMT | AMANDA/Hardware | | permalink

SPADE tape tape_2007_368_RAW_copy_1 full. Inserted new tape.


Ethan Dicks | 30 Nov 2007 17:40 GMT | SPADE | | permalink

Reconnected GPS cable to Rack 6

See more ...


Ethan Dicks | 30 Nov 2007 15:42 GMT | Ice Cube/SPS | | permalink

Nov 29, 2007

sps-sattx:/dev/nst2 needs a new tape

See more ...


Edgar Nielsen | 29 Nov 2007 21:10 GMT | SPADE | | permalink

sps-sattx:/dev/nst0 needs a new tape

See more ...


Edgar Nielsen | 29 Nov 2007 21:03 GMT | SPADE | | permalink

sps-sattx:/dev/nst1 needs attention

See more ...


Edgar Nielsen | 29 Nov 2007 02:08 GMT | SPADE | | permalink

Nov 28, 2007

sps-amanda does not appear to be on the network at this time

See more ...


Edgar Nielsen | 28 Nov 2007 22:16 GMT | AMANDA/Hardware | | permalink

Andreas Gross requests power cycle of TWR crate 0

See more ...


Edgar Nielsen | 28 Nov 2007 16:22 GMT | AMANDA/Hardware | | permalink

Weekly run

See more ...


Edgar Nielsen | 28 Nov 2007 05:21 GMT | Ice Cube/Runs | | permalink

Nov 27, 2007

sps-build32:/dev/nst1 needs attention

See more ...


Edgar Nielsen | 27 Nov 2007 21:38 GMT | SPADE | | permalink

SPADE tape tape_2007_366_RAW_copy_1 full. Inserted new tape from CV boxes.


Ethan Dicks | 27 Nov 2007 16:48 GMT | SPADE | | permalink

Cleaning ICL

See more ...


Edgar Nielsen | 27 Nov 2007 15:16 GMT | Ice Cube/Infrastructure | | permalink

Cleaning ICL again

See more ...


Edgar Nielsen | 27 Nov 2007 15:16 GMT | Ice Cube/Infrastructure | | permalink

Nov 26, 2007

DOMHubMonitor reports problems with sps-ithub01

See more ...


Edgar Nielsen | 26 Nov 2007 21:17 GMT | Ice Cube/Runs | | permalink

sps-build32:/dev/nst0 needs attention

See more ...


Edgar Nielsen | 26 Nov 2007 15:53 GMT | SPADE | | permalink

SUMMARY

See more ...


Edgar Nielsen | 26 Nov 2007 15:30 GMT | SPADE | | permalink

ic-tape:/dev/nst0 needs a new tape

See more ...


Edgar Nielsen | 26 Nov 2007 15:17 GMT | SPADE | | permalink

Altered pdaq crontab on sps-expcont

See more ...


Edgar Nielsen | 26 Nov 2007 03:53 GMT | Ice Cube/Infrastructure | | permalink

sps-sattx:/dev/nst3 needs a new tape

See more ...


Edgar Nielsen | 26 Nov 2007 03:30 GMT | SPADE | | permalink

Worked on enabling the event viewer software

See more ...


Edgar Nielsen | 26 Nov 2007 00:18 GMT | Ice Cube/Misc | | permalink

Morning work done for IceTop

See more ...


Edgar Nielsen | 26 Nov 2007 00:02 GMT | Ice Cube/Ice Top | | permalink

Nov 25, 2007

SPADE tape tape_2007_363_RAW_copy_1 full. Inserted new tape from "CV" boxes.


Ethan Dicks | 25 Nov 2007 14:24 GMT | SPADE | | permalink

Nov 24, 2007

sps-sattx:/dev/nst0 needs to be changed.

See more ...


Edgar Nielsen | 24 Nov 2007 02:00 GMT | SPADE | | permalink

Nov 23, 2007

sps-sattx:/dev/nst1 needs attention

See more ...


Edgar Nielsen | 23 Nov 2007 15:10 GMT | SPADE | | permalink

Nov 22, 2007

More training by Team AMANDA

See more ...


Edgar Nielsen | 22 Nov 2007 20:59 GMT | AMANDA/Training | | permalink

sps-build32 /dev/nst1 needs attention

See more ...


Edgar Nielsen | 22 Nov 2007 20:58 GMT | SPADE | | permalink

Helped IceTop install DOMs in tanks

See more ...


Edgar Nielsen | 22 Nov 2007 18:45 GMT | Ice Cube/Ice Top | | permalink

Ethan showed me how to configure the dAOM settings for strings 18 and 19

See more ...


Edgar Nielsen | 22 Nov 2007 17:14 GMT | AMANDA/Hardware | | permalink

We are splitting off quad 9(c3p2+3) on sps-ichub59

See more ...


Edgar Nielsen | 22 Nov 2007 14:39 GMT | Ice Cube/DOMs | | permalink

sps-build32:/dev/nst1 needs a new tape

See more ...


Edgar Nielsen | 22 Nov 2007 14:36 GMT | SPADE | | permalink

sps-sattx:/dev/nst1 needs attention

See more ...


Edgar Nielsen | 22 Nov 2007 13:11 GMT | SPADE | | permalink

sps-build32:/dev/nst0 SPADE Alert - java.rmi.RemoteException

See more ...


Edgar Nielsen | 22 Nov 2007 03:05 GMT | SPADE | | permalink

Nov 21, 2007

SPADE tape tape_2007_358_RAW_copy_1 full. Inserted new tape from "CV" boxes.


Ethan Dicks | 21 Nov 2007 17:26 GMT | SPADE | | permalink

While doing power logs, found a variety of issues

See more ...


Edgar Nielsen | 21 Nov 2007 15:37 GMT | Ice Cube/Infrastructure | | permalink

sps-build32:/dev/nst0 needs attention

See more ...


Edgar Nielsen | 21 Nov 2007 15:09 GMT | SPADE | | permalink

Latest news on the redhat 4.5 twrdaq machines.

See more ...


Edgar Nielsen | 21 Nov 2007 03:29 GMT | AMANDA/Hardware | | permalink

Nov 20, 2007

Weekly runs being run

See more ...


Edgar Nielsen | 20 Nov 2007 16:09 GMT | Ice Cube/Runs | | permalink

sps-sattx:/dev/nst1 needs a new tape

See more ...


Edgar Nielsen | 20 Nov 2007 14:04 GMT | SPADE | | permalink

ic-tape:/dev/nst0 needs attention

See more ...


Edgar Nielsen | 20 Nov 2007 13:31 GMT | SPADE | | permalink

ic-tape:/dev/nst1 needs a new tape

See more ...


Edgar Nielsen | 20 Nov 2007 12:30 GMT | SPADE | | permalink

Nov 19, 2007

Latest news on the redhat 4.5 twrdaq machines.

See more ...


Edgar Nielsen | 19 Nov 2007 19:12 GMT | AMANDA/Hardware | | permalink

sps-ichub72 acopian power supply fixed and original run configuration restored

See more ...


Edgar Nielsen | 19 Nov 2007 03:36 GMT | Ice Cube/Runs | | permalink

Drive error light on drive 1/1 in sps-build32.

See more ...


Ethan Dicks | 19 Nov 2007 02:33 GMT | Ice Cube/SPS | | permalink

Drive error light on drive 0/0 in sps-build64.

See more ...


Ethan Dicks | 19 Nov 2007 02:32 GMT | Ice Cube/SPS | | permalink

Failed Acopian DOM PSU replaced in ic-hub72

See more ...


Ethan Dicks | 19 Nov 2007 01:44 GMT | Ice Cube/SPS | | permalink

Nov 18, 2007

DOM PSU failed on ic-hub72 - restarted pDAQ with new config

See more ...


Ethan Dicks | 18 Nov 2007 23:12 GMT | Ice Cube/Runs | | permalink

SPADE tape tape_2007_352_RAW_copy_1 full. Inserted new tape from "CV" boxes.


Ethan Dicks | 18 Nov 2007 21:25 GMT | SPADE | | permalink

Disabling tape drives on ic-tape due to network work

See more ...


Edgar Nielsen | 18 Nov 2007 00:15 GMT | SPADE | | permalink

Nov 17, 2007

sps-sattx:/dev/nst2 needs attention

See more ...


Edgar Nielsen | 17 Nov 2007 17:40 GMT | SPADE | | permalink

sps-build32:/dev/nst1 needs attention

See more ...


Edgar Nielsen | 17 Nov 2007 17:40 GMT | SPADE | | permalink

rate monitor alert email arrived

See more ...


Edgar Nielsen | 17 Nov 2007 15:49 GMT | Ice Cube/Runs | | permalink

A rate monitor alert email arrived; later more came in

See more ...


Edgar Nielsen | 17 Nov 2007 04:13 GMT | Ice Cube/Runs | | permalink

Nov 16, 2007

Afternoon IceTop Work

See more ...


Edgar Nielsen | 16 Nov 2007 22:12 GMT | Ice Cube/Ice Top | | permalink

More status on the ic-twrdaq machine

See more ...


Edgar Nielsen | 16 Nov 2007 18:31 GMT | AMANDA/Hardware | | permalink

Found 2 acopian power supplies on

See more ...


Edgar Nielsen | 16 Nov 2007 18:22 GMT | Ice Cube/Ice Top | | permalink

Work I did for IceTop this morning

See more ...


Edgar Nielsen | 16 Nov 2007 17:41 GMT | Ice Cube/Ice Top | | permalink

Alert email on sps-build32:/dev/nst0

See more ...


Edgar Nielsen | 16 Nov 2007 15:22 GMT | SPADE | | permalink

Darryn having AMANDA Hub and synchronization box in ICL moved.

See more ...


Edgar Nielsen | 16 Nov 2007 01:19 GMT | AMANDA/Hardware | | permalink

Nov 15, 2007

Received an anvil alert email.

See more ...


Edgar Nielsen | 15 Nov 2007 18:33 GMT | Ice Cube/Runs | | permalink

SPADE tape tape_2007_349_RAW_copy_1 full. Inserted new tape.


Ethan Dicks | 15 Nov 2007 17:36 GMT | SPADE | | permalink

sps-build32:/dev/nst1 needs attention

See more ...


Edgar Nielsen | 15 Nov 2007 16:16 GMT | SPADE | | permalink

sps-sattx:/dev/nst1 tape change

See more ...


Edgar Nielsen | 15 Nov 2007 15:49 GMT | SPADE | | permalink

More work on the new ic-twrdaq machine

See more ...


Edgar Nielsen | 15 Nov 2007 01:38 GMT | AMANDA/Hardware | | permalink

ic-tape:/dev/nst1 needs a tape change

See more ...


Edgar Nielsen | 15 Nov 2007 01:27 GMT | SPADE | | permalink

Nov 14, 2007

Received a DOMHub Monitor Alert

See more ...


Edgar Nielsen | 14 Nov 2007 23:07 GMT | Ice Cube/Runs | | permalink

We extracted the remaining FCU's from the ice.

See more ...


Edgar Nielsen | 14 Nov 2007 21:50 GMT | Ice Cube/Ice Top | | permalink

More work on the new ic-twrdaq machine

See more ...


Edgar Nielsen | 14 Nov 2007 00:53 GMT | AMANDA/Hardware | | permalink

Rate Monitor alert

See more ...


Edgar Nielsen | 14 Nov 2007 00:08 GMT | Ice Cube/Runs | | permalink

Rate monitor alert

See more ...


Edgar Nielsen | 14 Nov 2007 00:08 GMT | Ice Cube/Runs | | permalink

Nov 13, 2007

SAM service error alert

See more ...


Edgar Nielsen | 13 Nov 2007 18:38 GMT | Ice Cube/Runs | | permalink

Helped remove the FCUs from 73[AB] and 55[AB]

See more ...


Edgar Nielsen | 13 Nov 2007 15:56 GMT | Ice Cube/Ice Top | | permalink

sps-sattx:/dev/nst3 needs attention

See more ...


Edgar Nielsen | 13 Nov 2007 15:52 GMT | SPADE | | permalink

sps-build32:/dev/nst1 needs attention

See more ...


Edgar Nielsen | 13 Nov 2007 14:05 GMT | SPADE | | permalink

Noticed that Acopian for 73[AB] in unusual state

See more ...


Edgar Nielsen | 13 Nov 2007 13:50 GMT | Ice Cube/Ice Top | | permalink

sps-sattx:/dev/nst0 needs a tape change

See more ...


Edgar Nielsen | 13 Nov 2007 13:45 GMT | SPADE | | permalink

SAM service error alert

See more ...


Edgar Nielsen | 13 Nov 2007 00:07 GMT | Ice Cube/Runs | | permalink

Nov 12, 2007

SAM service error alert

See more ...


Edgar Nielsen | 12 Nov 2007 21:11 GMT | Ice Cube/Runs | | permalink

Nov 11, 2007

More work troubleshooting ic-twrdaq

See more ...


Edgar Nielsen | 11 Nov 2007 20:03 GMT | AMANDA/Hardware | | permalink

sps-sattx:/dev/nst3 drive has clean LED blinking

See more ...


Edgar Nielsen | 11 Nov 2007 15:30 GMT | SPADE | | permalink

sps-build32:/dev/nst0 needs attention

See more ...


Ethan Dicks | 11 Nov 2007 15:07 GMT | SPADE | | permalink

Nov 10, 2007

sps-sattx:/dev/nst1 needs a new tape

See more ...


Edgar Nielsen | 10 Nov 2007 23:26 GMT | SPADE | | permalink

SPADE tape fix on sps-sattx:/dev/nst3

See more ...


Ethan Dicks | 10 Nov 2007 21:08 GMT | SPADE | | permalink

SPADE tape fix on sps-build32:/dev/nst1

See more ...


Ethan Dicks | 10 Nov 2007 21:08 GMT | SPADE | | permalink

Nov 09, 2007

SPADE tape fix on sps-sattx:/dev/nst2

See more ...


Ethan Dicks | 09 Nov 2007 22:11 GMT | SPADE | | permalink

Shutting down the HV Crates during Power Maintenance Window.

See more ...


Edgar Nielsen | 09 Nov 2007 21:43 GMT | AMANDA/Hardware | | permalink

Brought 4x300GB hard drives to MAPO from ICL spares

See more ...


Edgar Nielsen | 09 Nov 2007 15:43 GMT | AMANDA/Hardware | | permalink

SPADE tape fix on sps-build32:/dev/nst1

See more ...


Edgar Nielsen | 09 Nov 2007 15:09 GMT | SPADE | | permalink

SPADE tape fix on ic-tape:/dev/nst0

See more ...


Edgar Nielsen | 09 Nov 2007 14:12 GMT | SPADE | | permalink

Shutting down the HV Crates during Power Maintenance Window.

See more ...


Edgar Nielsen | 09 Nov 2007 13:34 GMT | AMANDA/Hardware | | permalink

Nov 08, 2007

SPADE tape fix on sps-sattx:/dev/nst0

See more ...


Edgar Nielsen | 08 Nov 2007 20:38 GMT | SPADE | | permalink

Nov 07, 2007

Two tapes to change and one to fix on SPADE

See more ...


Edgar Nielsen | 07 Nov 2007 20:27 GMT | SPADE | | permalink

Ensuring all setpoints to 25 degrees

See more ...


Edgar Nielsen | 07 Nov 2007 20:05 GMT | Ice Cube/Ice Top | | permalink

Two tapes to change and one to fix on SPADE

See more ...


Edgar Nielsen | 07 Nov 2007 19:33 GMT | SPADE | | permalink

Followup trip to ICL to follow SOP for sps-build32:/dev/nst1

See more ...


Edgar Nielsen | 07 Nov 2007 19:32 GMT | SPADE | | permalink

sps-build32:/dev/nst1 needs attention email

See more ...


Edgar Nielsen | 07 Nov 2007 19:31 GMT | SPADE | | permalink

Multiple SPADE tape issues addressed

See more ...


Edgar Nielsen | 07 Nov 2007 19:28 GMT | SPADE | | permalink

Potential Network Switch Error, sps-sw14:pGi1/14

See more ...


Edgar Nielsen | 07 Nov 2007 19:26 GMT | Ice Cube | | permalink

Shutting down the HV Crates during Power Maintenance Window.

See more ...


Edgar Nielsen | 07 Nov 2007 16:33 GMT | AMANDA/Hardware | | permalink

Nov 06, 2007

Helped configure twrdaq machine as not properly configured

By: TeX
Time: 01:00UTC - 03:39UTC (approx)

The new twrdaq machine needed the following items configured:
	Info based on data provided by Karthik(I wrote new script):
		Get ldap ssl certificates
		/etc/ldap.conf 
		/etc/openldap/ldap.conf
	I did:
		Configure /etc/nsswitch.conf
		Configure /etc/resolv.conf
		Configure networking setup
		Configure /etc/ntp.conf, /etc/ntp/step-tickers
		Configured sudoers file
		Set autofs to start at boot
		Set irqbalance to start at boot

System was last updated in September, up2date not working, either not
properly configured on the system or up2date system not working at ICL.

Edgar Nielsen | 06 Nov 2007 22:07 GMT | AMANDA/Hardware | | permalink

Due to power work, shutdown AMANDA, problems with DMADD at startup.

By: TeX, with Sven and Ethan
Time: 09:00NZ - 12:00NZ

Due to power work, we decided to shutdown AMANDA. Normally would have
just shutdown the non-UPS components, but for training purposes we
shutdown everything so we could also vett the documentation.

AMANDA was already standalone, and not running.

Things to fix:
	*) WOs need to be in hv group to run the scripts
	*) none of the machines power off after shutdown.
	*) systems do not auto shutdown if UPS goes on battery.
	*) Some documentation updates.

After bringing everything up, unable to run check_and_initialize_dmadd.pl
without errors about DISC_108. Mail sent to Ralf about this as
repeated power cycles and runs of the script produced identical
results.

Went to lunch after firing off eventbuilder.


Edgar Nielsen | 06 Nov 2007 22:07 GMT | AMANDA/Hardware | | permalink

Work with Andreas T. on the DMADD problem

By: TeX
Time: 03:00UTC

Andreas was attempting to look at run output to see if OK.
Had issues as data being xferred via SPADE. He fired off a new run.

03:47UTC
After much finagling with his code - I think it is for reading the output -
AT states that TWR DAQ appears to be fine, but issues with DMADD still exist.

03:59UTC
AT states the 50Mhz is having issues, has o do more debugging.

Edgar Nielsen | 06 Nov 2007 22:00 GMT | AMANDA/Hardware | | permalink

Nov 05, 2007

Round Two Checking the Temperatures of the FCUs (as per the SOP)

See more ...


Edgar Nielsen | 05 Nov 2007 19:31 GMT | Ice Cube/Ice Top | | permalink

Raising the setpoint to 25C for half of the tanks on each acopian, started

around 03:00UTC.

See more ...


Edgar Nielsen | 05 Nov 2007 19:21 GMT | Ice Cube/Ice Top | | permalink