There are now two extraction servers running new version of XML-RPC InteractiveServer: lxvm0341:10001 and voatlas18:10001.

Servers can be accessed using standard tools, via command line on CERN AFS:

# Get Java and Athenaeum
source /afs/cern.ch/sw/lcg/external/Java/bin/setup.sh
# Get list of available tasks
athenaeum
# Get Server Help
athenaeum http://voatlas18.cern.ch:10001 insider Help
# Get environment of the running Server
athenaeum http://voatlas18.cern.ch:10001 insider Info
# Get log of recent tasks
athenaeum http://voatlas18.cern.ch:10001 insider Log

or via Athenaeum Web Service:

• Server Help
• Environment of the running Server
• Log of the recent tasks

Updated documentation is available at Server setup and Client example Wiki pages.

The XML-RPC server implementation (together with managemetn and testing tools) is available from the Database/TAGPoolServices CVS package. There are two important changes with respect to the original server provided within Atlantis:

• The remore message is gzipped by the modified SimpleXMLRPCServer.py file.
• The server is multithreaded, it can server several clients in parallel. The executed scripts should take that into account while handling global variables and environment. All stdout and stderr messages go into server log files. All returns from the functions should be passed using answer global array and process pid, like this:

  def MyFunction(pid,...)
    resultMsg = ""
    ...
    resultMsg += "done for " + pid
    global answer
    answer[pid] = resultMsg

  The calling function should specify pid formal argument, it will be filled by Athenaeum:

  MyFunction('pid',...)

I have finally managed to convert MPEG files from movie camera into DVD. Camera exports only NTSC (those things are so cheaper in USA today :-) so direct connection is useless in Europe. The Windows software coming with the camera is very clumsy. The Linux software, on the other hand, can do the job very well. It just takes some time to find out. Here is, what seems to work fine:

# mpeg files can be just concatenated
cat *mpg > all.mpg

# mpeg should be converted to DVD-style mpeg (very slow)
#   yadif=3 ... deinterlace
#   -ofps=30000/1001 ... don't skip frames
#   mbd=2 ... slower, but better
mencoder \
-oac lavc \
-ovc lavc \
-of mpeg \
-mpegopts format=dvd \
-vf yadif=3,mcdeint=1,framestep=2,scale=720:576,harddup \
-srate 48000 \
-af lavcresample=48000 \
-lavcopts vcodec=mpeg2video:mbd=2:vrc_buf_size=1835:vrc_maxrate=9800:vbitrate=5000:\
keyint=15:aspect=16/9:acodec=ac3:abitrate=192 \
-ofps 30000/1001 \
-o dvd.mpg all.mpg

# DeVeDe creates dvd.iso
devede

# K3b writes dvd.iso to DVD
k3b

Athenaeum Clients (command-line, JAS and Web Service) call Python functions on  Athenaeum Servers. Some functions are server specific, that means that they can run only on dedicated servers as they depend on the server environment. Clients may access Server functions via multiple front-ends, using specific Client scripts. Some Clients may not support some Server functions.

Standard Athenaeum distribution supports three functions:

• Cool allows to browse Cool DB using Athena Server. You can connect to any Cool database by filling connection parameters. Its functionality is implemented by the Client script.
• Event gives ntuple representation of the current Athena event. Its functionality is implemented by the Client script.  (currently only available via JAS client)
• Extract extracts a ROOT Collection with all references and all metadata from a relational TAG database for use as input to an analysis job. It is implemented by the Extract(...) function.  (currently only available via Web Service client on the ELSSI Gateway)

Other functions are general, they can be implemented on any server:

• Help gives help from running Server, if implemented by the help() funcion.
• Restart restarts running Server, if implemented by the restart() function. User should wait about a minute before sending new requests to the Server.
• Fork starts new (personal) Server on chosen port, if implemented by the fork(port) function. User should wait about a minute before sending new requests to the Server.

Some functions are available on all servers, they don’t need any special implementation:

• Info gives basic information about running Server.
• Log gives log of latest task on the selected server:port. (depends on http server, which may be unavailable or hidden behind a firewall)
• Dummy allows to run any Python Script on the Athena Server.

Standard Server script should look like this:

#--------------------------------------------------------------
# Define all callable functions and environment
#--------------------------------------------------------------
def abc():
  print "abc"

#--------------------------------------------------------------
# Define Help
#--------------------------------------------------------------
def help():
  print "This is the help !"

#--------------------------------------------------------------
# Define Restarting
#--------------------------------------------------------------
def restart():
  os.spawnlp(os.P_NOWAIT, "athena.py", "athena.py", "-i", "-s", "Server.py")

#--------------------------------------------------------------
# Define Forking
#--------------------------------------------------------------
def fork(port):
  os.environ["ATHENAEUM_PORT"] = port;
  os.spawnlp(os.P_NOWAIT, "python", "python", "Server.py")

#--------------------------------------------------------------
# Start the Server
#--------------------------------------------------------------
argumentInteractiveServerNoAlgTools = True
argumentInteractiveServerServerPort = os.environ["ATHENAEUM_PORT"]
argumentInteractiveServerServerKey = "mykey"
execfile ("InteractiveServer.py")

Server script should be in the same directory as Interactive Server scripts. They are available from the Athenaeum distribution directory src/Server/Common.

Server script can be started (usually from cron) by a simple shell script like this:

# Define server port
export ATHENAEUM_PORT="11111"
# Start Http server, if remote access to log files is required
python -u StartHttpServer.py &
# Start XML-RPC server itself
python -u StartServer.py

Some examples of the complete server setup are available from the Athenaeum distribution directory src/Server.

More information about Athenaum is available from its Home Page, Web Service and New Athenaeum command available blog entry.

GraXML has been mentioned in the CERN Computing Newsletter article CERN receives Duke’s Choice Award for its Java development:

The third application was the GraXML event viewer made by a Java developer in the ATLAS collaboration – Julius Hrivnac from the PH department. The tool permits a 3D visualization of the Atlas detector geometry with event (collision) data overlaid. With simple movements of the mouse, the detector can be manipulated in 3D space and the reconstructed particle tracks clearly seen as they traverse the various components of the detector.

There are more details about the Award (including video of the presentation) in GraXML in JavaOne post.

> One of the bugs I was fixing was in an inline function, defined in the header. The bug fix was tiny – changed one line of code. Perfect for a patch release. Unfortunately, a number of other packages used this inline function. And so they had to be rebuilt as well.

The problem is much deeper. There is not really any separation between interface and implementation in C++. You are obliged to expose parts of your implementation (things which should be hidden) in header files, which are seen by everyone else. One consequence is that you should recompile everything even when interface doesn’t change (as in your case). Another consequence is very long compilation time; compiler compiles in fact much bigger files than you implementation – it includes headers from many other classes.

> Ironically, however, at least in ATLAS, the compile step is fast — everything that has to be done to get to the compile step, however, is very slow (the build system).

The compile step is fast in comparison with other steps. However, the compile step in Atlas/C++ is incredibly slow if you compare it with some other languages. I have Java projects, where I compile several hundred classes in several seconds.

In fact, slowness of other steps comes probably mostly from the same source. Your end-user class may be very simple, but it depends on just everything else via repetitive include files. So the system has too get them first even before it starts doing anything serious. And when you are on AFS, it is already the amount of data you have to read (i.e. to move to your machine) which kills you. The same when you try to create libraries, build dictionaries,…

Even when your project may look very modular, it is not modular at the system level.

(from the dicsussion on the blog of Gordon Watts)

1. Get Database/AthenaeumServer package
2. Get all other packages you need so that your Athena runs fine and doesn’t crash
3. Get the latest Cool.py script from /afs/cern.ch/sw/lcg/external/Java/Athenaeum/<latest>/share/src/net/hep/atlas/Core/Athenaeum/JAS3Plugin/Proxies/Cool.py
4. Get some testing parameter files from /afs/cern.ch/sw/lcg/external/Java/Athenaeum/<latest>/share/src/net/hep/atlas/Core/Athenaeum/Test
5. Setup your Athena environment
6. Run athena.py StartAthenaServer0.py <param file> Cool.py

Alternatively, you may use client-server mode, as is described in another post.