ePubs

The open archive for STFC research publications

Full Record Details

Persistent URL http://purl.org/net/epubs/work/29842624
Record Status Checked
Record Id 29842624
Title ICAT Job Portal : a generic job submission system built on a scientific data catalog
Contributors
Abstract The value of metadata to the scientist is well known: with the right choice of metadata, data files can be selected very quickly without having to scan through huge volumes of data. The ICAT metadata catalog (which is part of the ICAT project) allows the scientist to store and query information about individual data files and sets of data files as well as storing provenance information. This paper explains how a generic job management system, exposed as a web portal, has been built on top of ICAT. This gives the scientist easy access to a high performance computing infrastructure without allowing the complexities of that infrastructure to impede progress. The aim was to build a job and data management portal capable of dealing with batch and interactive work that would be simple to use and that was based on tried and tested, scalable, and preferably open source technologies. For the team operating the portal, it needed to be generic and configurable enough so that they can, without too much effort, modify their software to run within the portal, add new software, and create new dataset types and parameters. Modifications to existing software should be limited to saving and loading their datasets in a slightly different way so that instead of just being saved to disk, they are registered within the system along with recording any provenance information.
Organisation STFC , SCI-COMP , SCI-COMP-DST
Keywords ICAT , portal , IDS , job , metadata
Funding Information
Related Research Object(s):
Licence Information:
Language English (EN)
Type Details URI(s) Local file(s) Year
Paper In Conference Proceedings In 5th International Workshop on Science Gateways (IWSG 2013), Zurich, Switzerland, 3-5 Jun 2013, CEUR-WS 993 (2013). http://ceur-ws.org/Vol-993/paper6.pdf 2013