NEOCC Tap Server Site Help

This document is intended to help you discover the web interface features on the NEOCC Tap Server. Note that the site also offers VO-enabled services. See, for example Getting Started on GAVO's web page for more information on how to use them.

Though most of the services should work even in text-based browsers, we recommend a web browser with CSS and Javascript enabled to access the web-based services.

Service discovery

The index page lists published services available through web browsers. A "[P]" in the service listing means that the service is password protected, either because it is too rough for public consumption or because the data providers want exclusive access. In either case, you can contact the site operators to inquire about access.

In addition to web-based services, the data center also provides services accessible through IVOA standard protocols like SIAP (for images), SCS (allowing simple queries against catalogs of astronomical objects), SSAP (for spectra), and TAP (allowing complex queries against database tables of all sorts). These can be located through Virtual Observatory registry interface (e.g., WIRR) or common VO tools.

Search forms

Query Fields

Services not using customised interfaces have standard query forms. These consist of a series of input fields that either take

• numeric expressions – you can recognize those from the little "[?num. expr.]" tag behind them. In addition to raw numbers, you can enter Vizier-like numeric expressions here.
• string expressions – these have a little "[?char expr.]" tag and by default match using patterns, evaluating metacharacters like * or ? much like you may know from file name patterns. To force literal matching, prepend your strings with ==. Other operators available for string expressions include caseless matching, string comparisons, or negation.
• date expressions – these are marked with "[?date expr.]" tags. Dates must be given in one of the VO's preferred ISO formats (YYYY-MM-DD or YYYY-MM-DDTHH:MM:SS). Among the more useful of the supported operators is the range – you can say 2004-01-02 .. 2005-05-01 to specify a range of dates between Jan 2nd, 2004 and May 1st, 2005; also note that dates without times (or datetimes with hour, minute, and second all 0) are sometimes treated specially (e.g., a simple comparison is expanded into a between) to match the behaviour of the backend database with the presumed user intention.
• Selection boxes – these are either drop down, in which case you can only select one entry, or open boxes you can select more than one entry in, usually (depending on your user interface) using control-click.
• Others – application-specific input fields (like, e.g., cone searches) should come with a short explanation.

Query Modifiers

In most forms, you will have a "Table" query field near the bottom of the form. You can set sorting and limit options there. Note that depending on the query, selecting a column to sort on may slow down the answer dramatically. This is because your query may match large amounts of data, and even if only 100 items are returned, potentially millions of them may have to be sorted.

On the other hand, results overflowing the match limit are not reproducible without a sort option, i.e., you may get a different set of, say, 100 items for identical query parameters at two different times. The services warn you about this fact when they return truncated results.

You can usually select an output format. Options here include

• HTML – data is returned in your web browser. You can select additional columns in your output from an input field that pops down when you mouse over it.
• VOTable – data is returned in IVOA's standard data format, the VOTable. This is XML that can be human-readable but is really intended to be consumed by tools like TOPCAT. You can select a "verbosity" specifying the fields present in the output, with 1 standing for a minimal set of information, 2 for what the service author deemed useful for the average astronomer, 3 for (almost) all fields available, and finally H for (essentially) what the default HTML table gives. Furthermore, you can choose between a "human-readable" VOTable (select this to process your data with standard XML or text-based tools or peruse it with the naked eye) and a binary version that you should use for larger data sets since it is much more efficient.
• FITS – this returns FITS tables. The data is in the first extension. This contains a good deal less meta information than a VOTable of the same data and thus should only be used if your backend tools do not understand VOTables or if the ability to memory-map the data contents matters in your application.
• TSV – tab separated files. If in a desperate pinch, you can get the table contents as an ASCII file. The fields a separated by tabs. All metadata is lost. Nullvalues are (almost) always rendered as the string "None". Strings containing non-ascii or control characters are rendered with C escapes (\n, \t, etc) or sedecimal unicode codepoints (\xe4, e.g., is an ). As I said: Don't use this unless you absolutely have to.
• CSV – comma separated values. This format carries almost no metadata as well, but it is understood by may database programs, spreadsheets, etc (note: don't use Excel to process astronomical data. TOPCAT is so much nicer, and it has built-in VO support). Null values are mostly rendered as empty fields, but float NULLs are NaNs.
• JSON – the JSON returned by DaCHS mimics the VOTable in structure and contains relatively rich metadata. Use this if you want to process query results with Javascript or in environments that do not understand VOTable.
• tar – if the service contains "products" (usually FITS files of some kind), you can download all matching items in a tar file.

Next to the output format selection, there are two links. The one labeled "[Result link]" reflects your current query as a copyable URL. You could use this to send a query result to colleagues, but it is particularly useful with the VOTable output format – you can drag and drop that link into TOPCAT's "Open" dialogue, for example.

The star () lets you bookmark the current form – it is basically like the result link but doesn't actually perform the query. This should come in handy to store and communicate preset queries. Note that for split-up fields (e.g., date inputs with year, month and day) you need to enter all pieces of information for this to work.

Both the result and the bookmark links require Javascript to work.

The Sidebar

The sidebar on form and result pages contains ancillary information for the resource the data comes from. To save space, the individual items are "folded in" by default. Mouse over them to see the full entries.

The "Created" metadata item shown refers to the creation of the resource descriptor, i.e., is related to the service rather than the data. The "Data updated" item gives the date of the last "import" of a piece of the data. Unless a service supports online modification, it can be used as an indicator of the version of the underlying data.

Timeouts

With ADQL, it is not hard to construct queries that will run for days or weeks, and even normal web-based queries can select so much data that excessive runtimes and possibly huge data volumes result. Usually, this is not intended by the users, and thus by default relatively short timeouts are imposed on the run time of the queries on the server (of the order 10 seconds).

You can usually increase the timeout by generating a link to your query (see "Result link" above) and appending "&_TIMEOUT=100 (or whatever number of seconds suits you) to the URL. The ADQL form has an input field that does just this.

However, usually long-running queries indicate that we should be adding some index to our tables. Thus, please contact us with your queries and we will try to speed them up.

Login and Logout

In the sidebar, you will usually see a login item. You should normally not need to be concerned with this, since protected services will query your credentials as needed. You may want to log out, though; if you click on the "log out <user>" link, a login mask will pop up. Leave the username blank and click OK to log out. If you click Cancel (or equivalent), you will remain logged in. This slight weirdness is inevitable when using HTTP authentication with current browsers.

There is one situation in which you will want to log in: Access to protected products from open services. You will note that in such cases, the system will deny access to the protected products without asking your credentials. If you can provide those, log in and try again.

To obtain usernames and passwords, contact neocc@esa.int.

Software availability

The software driving this site is available for download and inspection. For details see the DaCHS info page. All our code is released under the GPL.

Acknowledgements

This software uses services provided by the Centre de Données astronomiques de Strasbourg, France.

Some of the services use q3c indexing by Sergey Koposov and Oleg Bartunov, see 2006ASPC..351..735K.