Remote Experiments at BioMAX

Remote users should be familiar with the correct procedures to ship samples to and from BioMAX, with MXCuBE3, ISPyB and the computing environment at MAX IV, and the tools for handling the data and for remote access. New and non-frequent users should communicate in advance with staff to arrange remote or onsite training on those subjects before their beamtime.

Full remote access implies that the users must be able to mount and dismount samples, collect, process and backup the data without being present at MAX IV. The following requirements apply:

• The samples must be mounted on pins (spine pins are preferred) of the standard length (22 mm from base to tip) and loaded in UniPucks.
• Standard data collection at 100 K using collection modes available through MXCuBE3.  We are trying to implement support for remote data collection for some room temperature and fixed target serial crystallography experiments. Please contact the beamline staff if you are interested in collaborating in these developments.
• Data must be transferred through the internet trough SFTP or Globus only: copying data to external disks is not supported.
• Experiments from hazardous samples or any experiment that requires special instrumentation or setup require user presence on-site. Hybrid remote access may be granted to eligible collaborators outside MAX IV as long as one or more members of the proposal are present at the beamline.

Support staff will load and unload the UniPucks into the sample exchanger dewar and transport the dewar to the outshipping area, but will not be participate in the experiment beyond normal support duties (eg, providing basic scientific support and solving beamline issues which affect data collection). Any extensive involvement of staff in tasks which would be normally be done by users (transfer samples shipped in ESRF pucks, manual sample mounting or dealing with sample issues) should be agreed before hand and treated as a collaboration.

MAX IV users can access a dedicated BioMAX remote desktop for data collection or a HPC cluster for data processing after establishing a VPN connection to the beamline network. In order to establish a remote connection to MAX IV computers the following requirements apply:

• You must be a member of a proposal: either you are listed in the proposal as a PI or coproposer or included in a future or past experimental session for the proposal. New users who would like to have remote access before their beamtime has been schedule (e.g., for training or preparation purposes) should contact the beamline staff.
• Two-step authentication is compulsory to establish a remote VPN connection: After entering the DUO user name and password, you will be prompted for a one-time-password (otp). To set up the otp generator, please visit https://otp.maxiv.lu.se. As of 2024 it is no longer possible to get an otp via SMS.
• Users need the Ivanti Secure Access VPN Client (named Pulse Secure before 2022) or the Linux  OpenConnect client  and the Cendio ThinLinc client to connect to the beamtime remote server or the offline HPC cluster.

To install the software need for the remote connection, follow these steps:

• Follow the  instructions to download and install the VPN client.
• Next, download the Thinlinc client and install it.
• Test that the software installation has been successful by connecting to the offline data processing cluster front end, which is available all the time to all users listed in an active proposal:
  • Open the Pulse Secure application and connect to the vpn-white server; when prompted, select the DUO User realm, enter your DUO user name and password and the otp.
  • Connect to the Thinlinc client offline-fe1 by following these instructions. If the connection fails, see the “Troubleshooting” section below.

To be able to collect data at the beamline, you need to connect to the beamline network via a specific VPN server called vpn-blue.maxiv.lu. Access to the beamline VPN network is granted automatically at the start of the beamtime, provided that you are listed in the DUO experimental session.
To configure the VPN client for the beamline network:

• Open the Ivanti client and click on the + sign to add a new connection. Choose any name for the connection (e.g., “BioMAX”). The name of the server is vpn-blue.maxiv.lu.se. Ensure that there are no VPN connections open to any other server, including the VPN white network.
• When you click on the Connect button on the beamline network, you will get a long list of “Realms”. Scroll down to find BIOMAXremoteUser and click on the name to select it. Check the Save settings button if you want to store the connection details for future connections.
• Enter your DUO user name and password. After authenticating, type the otp code.

Once you are connected to the beamline VPN, open the Thinlinc client, and connect to the server biomax-remote.

See IT environment at BioMAX for more information about the remote desktop.

For remote experiments, ship the dewar containing UniPucks to MAX IV as usual. It is advisable to ship the dewar to get to MAX IV one or two days before the experiment at the latest. Please consult the EXI documentation for help with organizing the shipment.

On the beamtime day, the beamline staff will load the UniPucks in the dewar and  inform you when the hutch is searched and you can start the experiment. They will also provide you with contact information in case you have questions during the experiment or run into problems.

To start the experiment, open the VPN connection to the beamline network as described in the previous section. Then, open Thinlinc, enter the beamline server name (biomax-remote) and your login credentials. The remote desktop is configured to look like the workstations at BioMAX and is part of the same computing environment. Links to MXCuBE3 and EXI and software to view diffraction images and monitor the beam position are available as icons on the desktop. See the MXCuBE documentation for more details.

Note: If you are a first-time user at BioMAX, there will be no MXCuBE or other software desktop icons the first time you log in. After you log out and log in again you will see the icons.

• To run MXCuBE3, we recommend using a screen of at least 1680×1050. Otherwise the GUI will not fit in the window, and it can become difficult to access all the buttons and links. The bottom right side of the GUI has a icon that links to video feeds from the BioMAX hutch. You can select between a wide angle view of the beamline, useful to monitor the sample mounting robot, or a closer view of the diffractometer.
• Because remote access enables participation in the experiment from collaborators at different locations, MXCuBE also supports features for multiuser access. Please see the MXCuBE remote access documentation.

• The data will be stored in the usual location and you will be able to display the autoprocessing results on ISPyB in the remote server. Note that MAX IV hosted pages and services may not be accessible on your computer while you are connected to the beamline VPN, but will be available on biomax-remote.
• If you wish to process the data manually during the experiment, connect to the online HPC cluster from the biomax-remote server via Thinlinc (in the top bar menu, select Applications -> Internet -> Thinlinc client. Use clu0-fe-2 or clu0-fe-3 as the server and log in with your DUO credentials).
• You will not be able to open a connection to the offline HPC on the same machine as the remote beamline server; you can only have one VPN connection open at one time.
• If you lose the connection temporarily, you will be able to reconnect to the Thinlinc session and resume the experiment; connection outages do not affect ongoing data collections or beamline tasks. Also, you can conduct the experiments at different locations (eg, you can start at work and then continue from home). If you do not log out from the Thinlinc session, you will be able to reconnect to it from any location. However, MXCuBE3 does not allow the same user to be logged in twice, so if you try to log in twice (for example, both from the beamline and remotely) you will not be able to connect unless you force-quit the first MXCuBE instance from the log in window.

Problems with the VPN connection

• After installing the VPN client, the first time connection may fail. Reboot the machine if this happens.
• Double check that you are logging in with the correct credentials; the login is case-sensitive. If at some point you save your credentials in the connection and then change the password, you will have to create a new connection to be able to log in again.
• If you can connect to the office network vpn-white, but not the beamline network vpn-blue, check that 1) there is an experimental session in DUO for your beamtime and 2) you are included in the current or a previous experimental session in your proposal; note that the permission to access the beamline network is not updated immediately, so if you have been added to the session just before or after the beamtime starts, it can take up to half an hour until you are actually enabled to open the VPN connection. Also make sure that you select the BIOMAXremoteUser realm, not the BIOMAX realm (which is reserved for staff).
• Make sure that you do not have another VPN connection on the same computer.
• The VPN client checks for updates so at some point you will get a message about updating the application after you log in. Accepting the message will terminate your current session; it is OK to ignore this message and do the update later.

Problems with the Thinlinc server

• If the server name cannot be resolved, check that the VPN connection is still on and that you are trying to log in to the correct server (eg, if you are connected to the vpn-white server, you will not be able to log in to the biomax-remote server. Sometimes the problem can be solved by typing the entire domain name for the server. e.g. biomax-remote.maxiv.lu.se
• If the login error is “server does not support authentication”, try using the IP address to log in: 172.18.116.20.
• If the remote desktop window displays only a black background, there may be too many processes running on the biomax-remote machine. Call staff.
• If you the connection fails to open an SSH tunnel to the server, you may have a corrupted old session running. Click on the “End existing session” checkbox.
• If the server has run out of licences, contact staff.
• The Thinlinc window is maximized by default. If this is a problem, use the F8 key to change this behaviour in the Thinlinc menu.
• If you close the Thinlinc window by mistake, simply log in again. You should be reconnected to the old session, as long as the “End existing session” checkbox is unchecked.

Problems during the experiment

• There are some recurrent issues at MXCuBE commonly observed by users, described in the  MXCuBE3 documentation. As a remote user you may be more likely to fail to click on some menu buttons (eg, the Remote access menu, or the signout button), if you are running your remote session on a laptop or desktop with a small screen. Resizing the browser window should solve this.
• The remote desktop has four working spaces by default (they are displayed, iconized, at the right bottom part of the Thinlinc window). This is useful because you can open different programs in different spaces (eg, you can run MXCuBE, Albula and ISPyB in different spaces to make more room for each GUI). If you drag a window off to a side you may move it to the adjacent space by accident. Double check this before trying to reopen or restart the window.
• If the MXCuBE GUI or another program becomes unresponsive, check if other software runs on the remote desktop or your local computer before contacting staff. If your computer crashes or you lose the connection, you should be able to reconnect to the remote session again.
• A couple of users have reported a change in the colour palette of the screen when connected to the remote desktop. The problem clears after rebooting the computer.