Troubleshooting

1. Lost beam (no flux) - make sure that all lights are green on PSS panel and there is beam in the ring (check TV screen). If the beam was lost in the ring, after the repair is done APS will announce “shutter permit is enabled” – after that, manually open station A shutter on a PSS panel and wait for 15 minutes for a monochromator to warm up.

If there is a red light “Faults Trips” – call the floor coordinator (2-0101) to inquire about this problem.

Call your support beamline personnel if none of the above brings the beam.


2. The most common problem during the MD2 operation is that the program MD2_comwrapper is dying either because of lost communications between ADSC and MD2 or because of user error. When MD2_comwrapper is not running the data collection is hindered. Should md2_comwrapper_server fails, you should wait ONE minute to md2_comwrapper_server autorestart, if not, manually restart MD2 software: Exit from MD2 control system (Menu-File -> Exit), wait until the window disappears. Kill MD2_RPC_comwrapper window, if it is present (check the taskbar). Double click on the desktop icon MD2_APS_v.203, wait until MD2 system is running (2-3 minutes) and MD2_RPC_comwrapper window should appear in a while and it should be minimized to the taskbar. 

Next on “auxiliary” computer in the CONSOLE script under “DATA COLLECTION” tab select “UTILITIES” and choose “Restart ADSC system”

3. The experimental station cannot be searched – Check that the red button on the yellow box is not pressed in (occasionally someone may lean and push it). Check PSS panel: If there is a red light “Faults Trips” – call the floor coordinator (2-0101) to inquire about this problem


4. ADSC software does not collect any data after pressing “collect” – may be again the problem of MD2_comwrapper not running , see #2 above. Restarting ADSC software may also help. Before restarting ADSC software please close any running ADSC software by selecting EXIT and confirming.

Next on “auxiliary” computer in the CONSOLE script under “DATA COLLECTION” tab select “UTILITIES” and choose “Restart ADSC system” – a new GUI will start after in a while

It may be necessary to check for any ccd daemons still running: Exit ADSC and type “ps –e | grep ccd” and “ps –e | grep det”. If you see any ccd/det processes still running –kill them (kill -9 process id#). Now re-start again ADSC software by typing double-clicking on the green icon “AutoADSC” and “Restart ADCS System” from CONSOLE.


5. Alarm is sounding and the detector is moving back because some foreign object went too close to the front of the detector. Report it to your support personnel and disable the alarm on the panel.


6. If any computer is slow or frozen, the corresponding software should be restarted. To close the frozen EXECUTER window, press Execution Options – Abort SCRIPT in the top menu (use Ctrl-Alt-Del only as a last resort to kill CONSOLE, (do not send any report to M$). To start a new CONSOLE script, double-click on the desktop icon called “EXEC_Intel”, click on “Execution Option” and choose “Execute Script”

The following software should be always running:

Master beamline computer: EXECUTOR - C:\CONHEAP\konzuu_full_redis.CXE script.

Auxiliary beamline computer: EXECUTOR- C:\CONHEAP\PHI_MD2_FULL_REDIS.CXE script;

MD2 operation computer: MD2_comwrapper windows (in the background) and main MD2 window.


7. Restarting the whole MD2 controller – this operation should be done ONLY after receiving authorization from your support personnel.

Switching off sequence:

1. Exit MD2 Control Program on the PC.

2. Go inside C hutch and locate the MD2 control box (back wall , upstream and behind the MD2).

3. Press the RED Motion Supply button to stop all motors.

4. Wait for 30 secs.

5. Press the Main Supply to OFF position.

Switching on sequnce:

1. Press the Main Supply to ON position.

2. Press the GREEN Motion Supply button to enable all motors.

3. Wait for 30 secs.

4. Start MD2 Control Software on the PC.


8. Restarting GALIL server

Login to 164.54.212.68 or go inside the hutch and look for the crate behind the detector;

Open a new shell and make sure galil is not running by typing “ps –e |grep galil”

Type “cd /usr/local/galil_rpc_server”, type “./galil_rpc_server &”

Check again if galil is running by typing “ps –e |grep galil”, close the terminal

Master and auxiliary EXECUTOR scripts and ADSC software should be restarted after restarting galil server.


9. List of running servers (in progress):

Terminal is behind the adsc detector ; the KVM switch is behind the monitor

164.54.212.67 – pc68_rpc_server and hei_rpc_server

164.54.212.68 – pc68_rpc_server and galil_rpc_server

164.54.212.66 (in the rack close to LN2 controller by Station A) – serial…, utility…, diamond…, and pc68_rpc_server


10. gpfs file system is not responding. -- In an emergency you may collect data directly to the local hard disk on the ADSC computer. Use directory /data/users/your_institution/. Only for a short period of time and your support person should be notified.


12. Full restart of the control software (only after receiving authorization from your support personnel

Shutdown order for full reset on 24ID-C


ADSC (Far left computer)

exit from GUI or type “ccdsys shutdown” at terminal

type two commands : “ps -e | grep ccd” and “ps -e | grep det” at terminal

kill any remaining ccd/det processes by typing “kill -9 ###” (### = process id, the 1st column of the ps... results)

PHI_MD2_FULL_REDIS.CXE on Auxiliary (3rd from left)

press “EXIT” button in GUI (bottom-right button panel,); or “ExecutionOptions”->”AbortScript”

Press “File”-> “Exit EXECUTOR”


ROBOT_C.CXE inside the Hutch (robot computer, KVM switch)

Press “EXIT” button in GUI window (bottom-middle button panel, top-right button)

Press “File”-> “Exit EXECUTOR”

KONZUU_full_redis.CXE on Master (Far right computer; to access this screen, at Auxiliary and press CTRL twice)

Press “EXIT” in GUI window (bottom-right button); or “ExecutionOptions”->”AbortScript”

Press “File”-> “Exit EXECUTOR”


MD2 GUI (MD2 computer ; 2nd from left)

Exit GUI by pressing on the Menubar : “File->Exit”'; it takes ~90 sec to close


Comwrapper Server (MD2 computer ; 2nd from left) (This may be stowed in the taskbar)

Exit by by pressing “Red Windows X” in upper right corner


Restart order for full reset on 24ID-C


MD2 GUI (MD2 computer ; 2nd from left)


Robot Server (Inside the hutch)

-Double-click on the icon called EXECUTOR (it may be called Exec_Intel)

Click on the File -> “ClearConsServFlag”, confirm. Don’t start any scripts for now.


KONZUU_full_redis.CXE on Master (Far right computer; to access sit at Auxiliary and press CTRL twice)

-Double-click on the icon called Exec_Intel (it may be called EXECUTOR)

-Click on the File -> “ClearConsServFlag”


ROBOT_C.CXE inside the Hutch


PHI_MD2_FULL_REDIS.CXE on Auxiliary (3rd from left)

- Double-click on the icon called Exec_Intel

- Execute script by pressing on the Menubar : “Execution Options>>Execute Script”


ADSC (Far left computer)

- double-click on green diamond-like icon “AutoADSC”; wait for terminal with green fonts to appear

- On auxiliary computer go to “DATA COLLECTION” tab, select “UTILITIES” and choose “Restart ADSC system” – a new GUI will start after in a while

13. No images are coming to "ADXV" : Pilatus detector software needs to be re-started


i - On KVM switch press button #4
ii - On the middle monitor press ¤/¤
iii - kill the black window in left bottom corner
iv - check in another terminal : "ps -e | grep cam" -- if any "cam" process is present - kill it
v - Type "./camonly" and wait until all output in a new black window stops (~one minute)
vi - In a black window type "setenergy 12658" (or any energy the beamline is tuned now); hit "enter"; wait
vii - On KVM switch press button #4 and on the middle monitor press ¤/¤
viii - Restart EXECUTOR: 1)- exit, 2) ExecutionOptions->Execute script



Table of Content | Computer Layout and Beamline Software | Robot Operations | Energy Change | EXAFS Scan | Crystal Centering and Data Collection  | Data Backup |