Download this as a Jupyter notebook!
Marvin Queries¶
This tutorial goes through a few basics of how to perform queries on the MaNGA dataset using the Marvin Query tool. Please see the Marvin Query page for more details on how to use Queries. This tutorial covers the basics of:
querying on metadata information from the NSA catalog
how to combine multiple filter and return additional parameters
how to perform radial cone searches with Marvin
querying on information from the MaNGA DAPall summary file
querying using quality and target flags
First let’s import some basics
[4]:
# we should be using DR15 MaNGA data
from marvin import config
config.release
# import the Query tool
from marvin.tools.query import Query
INFO: No release version set. Setting default to DR15
/Users/Brian/anaconda2/envs/marvindev/lib/python3.5/importlib/_bootstrap.py:222: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
return f(*args, **kwds)
/Users/Brian/anaconda2/envs/marvindev/lib/python3.5/importlib/_bootstrap.py:222: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
return f(*args, **kwds)
/Users/Brian/anaconda2/envs/marvindev/lib/python3.5/importlib/_bootstrap.py:222: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
return f(*args, **kwds)
Query Basics¶
Querying on Metadata¶
Let’s go through some Query basics of how to do a query on metadata. The two main keyword arguments to Query are search_filter and return_params. search_filter is a string representing the SQL where condition you’d like to filter on. This tutorial assumes a basic familiarity with the SQL boolean syntax needed to construct Marvin Queries. Please see the tutorial on SQL Boolean syntax to learn
more. return_params is a list of parameters you want to return in the query in addition to those used in the SQL filter condition.
Let’s search for all galaxies with a redshift less than 0.1. To specify our search parameter, redshift, we must know the database table and name of the parameter. In this case, MaNGA uses the NASA-Sloan Atlas (NSA) for redshift information. In the NSA catalog, the redshift is the z parameter of the nsa table, so our search parameter will be nsa.z. Generically, all search parameters will take the form table.parameter.
[5]:
# filter for galaxies with a redshift < 0.1
my_filter = 'nsa.z < 0.1'
[7]:
# construct the query
q = Query(search_filter=my_filter)
q
[7]:
Marvin Query(filter='nsa.z < 0.1', mode='remote', data_origin='api')
The Query tool works with a local or remote manga database. Without a database, the Query tool submits your inputs to the remote server using the API, rather than doing anything locally. We run the query with the run method. Your inputs are sent to the server where the query is built dynamically and run. The results of the query are returned via the API, parsed and converted into a Marvin Results object.
[8]:
# run the query
r = q.run()
# print some results information
print(r)
print('number of results:', r.totalcount)
Marvin Results(query=nsa.z < 0.1, totalcount=4275, count=100, mode=remote)
number of results: 4275
After constructing queries, we can run them with q.run(). This returns a Marvin Results object. Let’s take a look. This query returned 4275 objects. For queries with large results, the results are automatically paginated in sets of 100 objects. Default parameters returned in queries always include the mangaid and plateifu. Marvin Queries will also return any parameters used in the definition of your filter condition. Since we filtered on redshift, the redshift is automatically included.
[12]:
# look at the current page of results (subset of 10)
print('number in current set:', len(r.results))
print(r.results[0:10])
number in current set: 100
<ResultSet(set=1.0/428, index=0:10, count_in_set=10, total=4275)>
[ResultRow(mangaid='1-109056', plateifu='8077-6103', z=0.0473019),
ResultRow(mangaid='1-109073', plateifu='8078-12704', z=0.0435882),
ResultRow(mangaid='1-109081', plateifu='8077-12705', z=0.0438592),
ResultRow(mangaid='1-109112', plateifu='8078-1901', z=0.0249702),
ResultRow(mangaid='1-109152', plateifu='8154-12702', z=0.0281736),
ResultRow(mangaid='1-109167', plateifu='8078-1902', z=0.0246344),
ResultRow(mangaid='1-109234', plateifu='9193-1901', z=0.0409309),
ResultRow(mangaid='1-109244', plateifu='8080-12702', z=0.0267911),
ResultRow(mangaid='1-109250', plateifu='8080-1902', z=0.0383731),
ResultRow(mangaid='1-109270', plateifu='9193-12703', z=0.0252266)]
Finding Available Parameters¶
We can use the Query datamodel to look up all the available parameters one can use in the search_filter or in the return_params keyword arguments to Query. The Query datamodel contains many parameters, which we bundle into groups for easier navigation. We currently offer four groups of parameters, with remaining parametrs captured in an Other group. See the Query Datamodel in the docs for
more detailed info.
[10]:
# look up the available query datamodel groups
q.datamodel.groups
[10]:
[<ParameterGroup name=Metadata, n_parameters=11>,
<ParameterGroup name=NSA Catalog, n_parameters=158>,
<ParameterGroup name=ObsInfo, n_parameters=64>,
<ParameterGroup name=DAPall Summary, n_parameters=620>,
<ParameterGroup name=Other, n_parameters=134>]
Each group contains a list of QueryParameters. A QueryParameter contains the designation and syntax used for specifying parameters in your query. The full attribute indicates the table.parameter name needed as input into the search_filter or return_params keyword arguments to the Marvin Query. The redshift is a parameter from the NSA catalog. There are 158 parameters in the NSA Catalog group.
[9]:
# select and print the NSA parameter group
nsa = q.datamodel.groups['nsa']
nsa.parameters
[9]:
[<QueryParameter full=nsa.iauname, name=iauname, short=iauname, remote=iauname, display=IAU Name>,
<QueryParameter full=nsa.ra, name=ra, short=ra, remote=ra, display=RA>,
<QueryParameter full=nsa.dec, name=dec, short=dec, remote=dec, display=Dec>,
<QueryParameter full=nsa.z, name=z, short=z, remote=z, display=Redshift>,
<QueryParameter full=nsa.elpetro_ba, name=elpetro_ba, short=axisratio, remote=elpetro_ba, display=Elpetro axis ratio>,
<QueryParameter full=nsa.elpetro_mag_g_r, name=elpetro_mag_g_r, short=g_r, remote=elpetro_mag_g_r, display=g-r>,
<QueryParameter full=nsa.elpetro_absmag_g_r, name=elpetro_absmag_g_r, short=absmag_g_r, remote=elpetro_absmag_g_r, display=Absmag g-r>,
<QueryParameter full=nsa.elpetro_logmass, name=elpetro_logmass, short=logmass, remote=elpetro_logmass, display=Elpetro Stellar Mass>,
<QueryParameter full=nsa.elpetro_th50_r, name=elpetro_th50_r, short=th50_r, remote=elpetro_th50_r, display=r-band half-light radius>,
<QueryParameter full=nsa.sersic_logmass, name=sersic_logmass, short=sersic_logmass, remote=sersic_logmass, display=Sersic Stellar Mass>,
<QueryParameter full=nsa.sersic_ba, name=sersic_ba, short=sersic_ba, remote=sersic_ba, display=Sersic axis ratio>,
<QueryParameter full=nsa.aid, name=aid, short=aid, remote=aid, display=Aid>,
<QueryParameter full=nsa.asymmetry, name=asymmetry, short=asymmetry, remote=asymmetry, display=Asymmetry>,
<QueryParameter full=nsa.bastokes, name=bastokes, short=bastokes, remote=bastokes, display=Bastokes>,
<QueryParameter full=nsa.camcol, name=camcol, short=camcol, remote=camcol, display=Camcol>,
<QueryParameter full=nsa.clumpy, name=clumpy, short=clumpy, remote=clumpy, display=Clumpy>,
<QueryParameter full=nsa.deccat, name=deccat, short=deccat, remote=deccat, display=Deccat>,
<QueryParameter full=nsa.dflags, name=dflags, short=dflags, remote=dflags, display=Dflags>,
<QueryParameter full=nsa.dversion, name=dversion, short=dversion, remote=dversion, display=Dversion>,
<QueryParameter full=nsa.elpetro_absmag, name=elpetro_absmag, short=elpetro_absmag, remote=elpetro_absmag, display=Elpetro_Absmag>,
<QueryParameter full=nsa.elpetro_absmag_F, name=elpetro_absmag_F, short=elpetro_absmag_F, remote=elpetro_absmag_F, display=Elpetro_Absmag_F>,
<QueryParameter full=nsa.elpetro_absmag_F_N, name=elpetro_absmag_F_N, short=elpetro_absmag_F_N, remote=elpetro_absmag_F_N, display=Elpetro_Absmag_F_N>,
<QueryParameter full=nsa.elpetro_absmag_F_g, name=elpetro_absmag_F_g, short=elpetro_absmag_F_g, remote=elpetro_absmag_F_g, display=Elpetro_Absmag_F_G>,
<QueryParameter full=nsa.elpetro_absmag_F_i, name=elpetro_absmag_F_i, short=elpetro_absmag_F_i, remote=elpetro_absmag_F_i, display=Elpetro_Absmag_F_I>,
<QueryParameter full=nsa.elpetro_absmag_F_r, name=elpetro_absmag_F_r, short=elpetro_absmag_F_r, remote=elpetro_absmag_F_r, display=Elpetro_Absmag_F_R>,
<QueryParameter full=nsa.elpetro_absmag_F_u, name=elpetro_absmag_F_u, short=elpetro_absmag_F_u, remote=elpetro_absmag_F_u, display=Elpetro_Absmag_F_U>,
<QueryParameter full=nsa.elpetro_absmag_F_z, name=elpetro_absmag_F_z, short=elpetro_absmag_F_z, remote=elpetro_absmag_F_z, display=Elpetro_Absmag_F_Z>,
<QueryParameter full=nsa.elpetro_absmag_N, name=elpetro_absmag_N, short=elpetro_absmag_N, remote=elpetro_absmag_N, display=Elpetro_Absmag_N>,
<QueryParameter full=nsa.elpetro_absmag_N_g, name=elpetro_absmag_N_g, short=elpetro_absmag_N_g, remote=elpetro_absmag_N_g, display=Elpetro_Absmag_N_G>,
<QueryParameter full=nsa.elpetro_absmag_N_i, name=elpetro_absmag_N_i, short=elpetro_absmag_N_i, remote=elpetro_absmag_N_i, display=Elpetro_Absmag_N_I>,
<QueryParameter full=nsa.elpetro_absmag_N_r, name=elpetro_absmag_N_r, short=elpetro_absmag_N_r, remote=elpetro_absmag_N_r, display=Elpetro_Absmag_N_R>,
<QueryParameter full=nsa.elpetro_absmag_N_u, name=elpetro_absmag_N_u, short=elpetro_absmag_N_u, remote=elpetro_absmag_N_u, display=Elpetro_Absmag_N_U>,
<QueryParameter full=nsa.elpetro_absmag_N_z, name=elpetro_absmag_N_z, short=elpetro_absmag_N_z, remote=elpetro_absmag_N_z, display=Elpetro_Absmag_N_Z>,
<QueryParameter full=nsa.elpetro_absmag_colour, name=elpetro_absmag_colour, short=elpetro_absmag_colour, remote=elpetro_absmag_colour, display=Elpetro_Absmag_Colour>,
<QueryParameter full=nsa.elpetro_absmag_g, name=elpetro_absmag_g, short=elpetro_absmag_g, remote=elpetro_absmag_g, display=Elpetro_Absmag_G>,
<QueryParameter full=nsa.elpetro_absmag_g_i, name=elpetro_absmag_g_i, short=elpetro_absmag_g_i, remote=elpetro_absmag_g_i, display=Elpetro_Absmag_G_I>,
<QueryParameter full=nsa.elpetro_absmag_g_z, name=elpetro_absmag_g_z, short=elpetro_absmag_g_z, remote=elpetro_absmag_g_z, display=Elpetro_Absmag_G_Z>,
<QueryParameter full=nsa.elpetro_absmag_i, name=elpetro_absmag_i, short=elpetro_absmag_i, remote=elpetro_absmag_i, display=Elpetro_Absmag_I>,
<QueryParameter full=nsa.elpetro_absmag_i_z, name=elpetro_absmag_i_z, short=elpetro_absmag_i_z, remote=elpetro_absmag_i_z, display=Elpetro_Absmag_I_Z>,
<QueryParameter full=nsa.elpetro_absmag_r, name=elpetro_absmag_r, short=elpetro_absmag_r, remote=elpetro_absmag_r, display=Elpetro_Absmag_R>,
<QueryParameter full=nsa.elpetro_absmag_r_i, name=elpetro_absmag_r_i, short=elpetro_absmag_r_i, remote=elpetro_absmag_r_i, display=Elpetro_Absmag_R_I>,
<QueryParameter full=nsa.elpetro_absmag_r_z, name=elpetro_absmag_r_z, short=elpetro_absmag_r_z, remote=elpetro_absmag_r_z, display=Elpetro_Absmag_R_Z>,
<QueryParameter full=nsa.elpetro_absmag_u, name=elpetro_absmag_u, short=elpetro_absmag_u, remote=elpetro_absmag_u, display=Elpetro_Absmag_U>,
<QueryParameter full=nsa.elpetro_absmag_u_g, name=elpetro_absmag_u_g, short=elpetro_absmag_u_g, remote=elpetro_absmag_u_g, display=Elpetro_Absmag_U_G>,
<QueryParameter full=nsa.elpetro_absmag_u_i, name=elpetro_absmag_u_i, short=elpetro_absmag_u_i, remote=elpetro_absmag_u_i, display=Elpetro_Absmag_U_I>,
<QueryParameter full=nsa.elpetro_absmag_u_r, name=elpetro_absmag_u_r, short=elpetro_absmag_u_r, remote=elpetro_absmag_u_r, display=Elpetro_Absmag_U_R>,
<QueryParameter full=nsa.elpetro_absmag_u_z, name=elpetro_absmag_u_z, short=elpetro_absmag_u_z, remote=elpetro_absmag_u_z, display=Elpetro_Absmag_U_Z>,
<QueryParameter full=nsa.elpetro_absmag_z, name=elpetro_absmag_z, short=elpetro_absmag_z, remote=elpetro_absmag_z, display=Elpetro_Absmag_Z>,
<QueryParameter full=nsa.elpetro_amivar, name=elpetro_amivar, short=elpetro_amivar, remote=elpetro_amivar, display=Elpetro_Amivar>,
<QueryParameter full=nsa.elpetro_apcorr, name=elpetro_apcorr, short=elpetro_apcorr, remote=elpetro_apcorr, display=Elpetro_Apcorr>,
<QueryParameter full=nsa.elpetro_apcorr_r, name=elpetro_apcorr_r, short=elpetro_apcorr_r, remote=elpetro_apcorr_r, display=Elpetro_Apcorr_R>,
<QueryParameter full=nsa.elpetro_apcorr_self, name=elpetro_apcorr_self, short=elpetro_apcorr_self, remote=elpetro_apcorr_self, display=Elpetro_Apcorr_Self>,
<QueryParameter full=nsa.elpetro_b1000, name=elpetro_b1000, short=elpetro_b1000, remote=elpetro_b1000, display=Elpetro_B1000>,
<QueryParameter full=nsa.elpetro_b300, name=elpetro_b300, short=elpetro_b300, remote=elpetro_b300, display=Elpetro_B300>,
<QueryParameter full=nsa.elpetro_colour, name=elpetro_colour, short=elpetro_colour, remote=elpetro_colour, display=Elpetro_Colour>,
<QueryParameter full=nsa.elpetro_flux, name=elpetro_flux, short=elpetro_flux, remote=elpetro_flux, display=Elpetro_Flux>,
<QueryParameter full=nsa.elpetro_flux_ivar, name=elpetro_flux_ivar, short=elpetro_flux_ivar, remote=elpetro_flux_ivar, display=Elpetro_Flux_Ivar>,
<QueryParameter full=nsa.elpetro_flux_ivar_r, name=elpetro_flux_ivar_r, short=elpetro_flux_ivar_r, remote=elpetro_flux_ivar_r, display=Elpetro_Flux_Ivar_R>,
<QueryParameter full=nsa.elpetro_flux_r, name=elpetro_flux_r, short=elpetro_flux_r, remote=elpetro_flux_r, display=Elpetro_Flux_R>,
<QueryParameter full=nsa.elpetro_kcoeff, name=elpetro_kcoeff, short=elpetro_kcoeff, remote=elpetro_kcoeff, display=Elpetro_Kcoeff>,
<QueryParameter full=nsa.elpetro_kcorrect, name=elpetro_kcorrect, short=elpetro_kcorrect, remote=elpetro_kcorrect, display=Elpetro_Kcorrect>,
<QueryParameter full=nsa.elpetro_mag_g, name=elpetro_mag_g, short=elpetro_mag_g, remote=elpetro_mag_g, display=Elpetro_Mag_G>,
<QueryParameter full=nsa.elpetro_mag_g_i, name=elpetro_mag_g_i, short=elpetro_mag_g_i, remote=elpetro_mag_g_i, display=Elpetro_Mag_G_I>,
<QueryParameter full=nsa.elpetro_mag_g_z, name=elpetro_mag_g_z, short=elpetro_mag_g_z, remote=elpetro_mag_g_z, display=Elpetro_Mag_G_Z>,
<QueryParameter full=nsa.elpetro_mag_i, name=elpetro_mag_i, short=elpetro_mag_i, remote=elpetro_mag_i, display=Elpetro_Mag_I>,
<QueryParameter full=nsa.elpetro_mag_i_z, name=elpetro_mag_i_z, short=elpetro_mag_i_z, remote=elpetro_mag_i_z, display=Elpetro_Mag_I_Z>,
<QueryParameter full=nsa.elpetro_mag_r, name=elpetro_mag_r, short=elpetro_mag_r, remote=elpetro_mag_r, display=Elpetro_Mag_R>,
<QueryParameter full=nsa.elpetro_mag_r_i, name=elpetro_mag_r_i, short=elpetro_mag_r_i, remote=elpetro_mag_r_i, display=Elpetro_Mag_R_I>,
<QueryParameter full=nsa.elpetro_mag_r_z, name=elpetro_mag_r_z, short=elpetro_mag_r_z, remote=elpetro_mag_r_z, display=Elpetro_Mag_R_Z>,
<QueryParameter full=nsa.elpetro_mag_u, name=elpetro_mag_u, short=elpetro_mag_u, remote=elpetro_mag_u, display=Elpetro_Mag_U>,
<QueryParameter full=nsa.elpetro_mag_u_g, name=elpetro_mag_u_g, short=elpetro_mag_u_g, remote=elpetro_mag_u_g, display=Elpetro_Mag_U_G>,
<QueryParameter full=nsa.elpetro_mag_u_i, name=elpetro_mag_u_i, short=elpetro_mag_u_i, remote=elpetro_mag_u_i, display=Elpetro_Mag_U_I>,
<QueryParameter full=nsa.elpetro_mag_u_r, name=elpetro_mag_u_r, short=elpetro_mag_u_r, remote=elpetro_mag_u_r, display=Elpetro_Mag_U_R>,
<QueryParameter full=nsa.elpetro_mag_u_z, name=elpetro_mag_u_z, short=elpetro_mag_u_z, remote=elpetro_mag_u_z, display=Elpetro_Mag_U_Z>,
<QueryParameter full=nsa.elpetro_mag_z, name=elpetro_mag_z, short=elpetro_mag_z, remote=elpetro_mag_z, display=Elpetro_Mag_Z>,
<QueryParameter full=nsa.elpetro_mass, name=elpetro_mass, short=elpetro_mass, remote=elpetro_mass, display=Elpetro_Mass>,
<QueryParameter full=nsa.elpetro_mets, name=elpetro_mets, short=elpetro_mets, remote=elpetro_mets, display=Elpetro_Mets>,
<QueryParameter full=nsa.elpetro_mtol, name=elpetro_mtol, short=elpetro_mtol, remote=elpetro_mtol, display=Elpetro_Mtol>,
<QueryParameter full=nsa.elpetro_nmgy, name=elpetro_nmgy, short=elpetro_nmgy, remote=elpetro_nmgy, display=Elpetro_Nmgy>,
<QueryParameter full=nsa.elpetro_nmgy_ivar, name=elpetro_nmgy_ivar, short=elpetro_nmgy_ivar, remote=elpetro_nmgy_ivar, display=Elpetro_Nmgy_Ivar>,
<QueryParameter full=nsa.elpetro_ok, name=elpetro_ok, short=elpetro_ok, remote=elpetro_ok, display=Elpetro_Ok>,
<QueryParameter full=nsa.elpetro_phi, name=elpetro_phi, short=elpetro_phi, remote=elpetro_phi, display=Elpetro_Phi>,
<QueryParameter full=nsa.elpetro_rnmgy, name=elpetro_rnmgy, short=elpetro_rnmgy, remote=elpetro_rnmgy, display=Elpetro_Rnmgy>,
<QueryParameter full=nsa.elpetro_th50, name=elpetro_th50, short=elpetro_th50, remote=elpetro_th50, display=Elpetro_Th50>,
<QueryParameter full=nsa.elpetro_th90, name=elpetro_th90, short=elpetro_th90, remote=elpetro_th90, display=Elpetro_Th90>,
<QueryParameter full=nsa.elpetro_th90_r, name=elpetro_th90_r, short=elpetro_th90_r, remote=elpetro_th90_r, display=Elpetro_Th90_R>,
<QueryParameter full=nsa.elpetro_theta, name=elpetro_theta, short=elpetro_theta, remote=elpetro_theta, display=Elpetro_Theta>,
<QueryParameter full=nsa.elpetro_theta_r, name=elpetro_theta_r, short=elpetro_theta_r, remote=elpetro_theta_r, display=Elpetro_Theta_R>,
<QueryParameter full=nsa.extinction, name=extinction, short=extinction, remote=extinction, display=Extinction>,
<QueryParameter full=nsa.fiber_flux, name=fiber_flux, short=fiber_flux, remote=fiber_flux, display=Fiber_Flux>,
<QueryParameter full=nsa.fiber_flux_ivar, name=fiber_flux_ivar, short=fiber_flux_ivar, remote=fiber_flux_ivar, display=Fiber_Flux_Ivar>,
<QueryParameter full=nsa.fiberid, name=fiberid, short=fiberid, remote=fiberid, display=Fiberid>,
<QueryParameter full=nsa.field, name=field, short=field, remote=field, display=Field>,
<QueryParameter full=nsa.ialfalfa, name=ialfalfa, short=ialfalfa, remote=ialfalfa, display=Ialfalfa>,
<QueryParameter full=nsa.in_dr7_lss, name=in_dr7_lss, short=in_dr7_lss, remote=in_dr7_lss, display=In_Dr7_Lss>,
<QueryParameter full=nsa.ined, name=ined, short=ined, remote=ined, display=Ined>,
<QueryParameter full=nsa.isdss, name=isdss, short=isdss, remote=isdss, display=Isdss>,
<QueryParameter full=nsa.isixdf, name=isixdf, short=isixdf, remote=isixdf, display=Isixdf>,
<QueryParameter full=nsa.itwodf, name=itwodf, short=itwodf, remote=itwodf, display=Itwodf>,
<QueryParameter full=nsa.izcat, name=izcat, short=izcat, remote=izcat, display=Izcat>,
<QueryParameter full=nsa.mag, name=mag, short=mag, remote=mag, display=Mag>,
<QueryParameter full=nsa.mangaTargets, name=mangaTargets, short=mangaTargets, remote=mangaTargets, display=Mangatargets>,
<QueryParameter full=nsa.mjd, name=mjd, short=mjd, remote=mjd, display=Mjd>,
<QueryParameter full=nsa.nprof, name=nprof, short=nprof, remote=nprof, display=Nprof>,
<QueryParameter full=nsa.nsaid, name=nsaid, short=nsaid, remote=nsaid, display=Nsaid>,
<QueryParameter full=nsa.petro_ba50, name=petro_ba50, short=petro_ba50, remote=petro_ba50, display=Petro_Ba50>,
<QueryParameter full=nsa.petro_ba90, name=petro_ba90, short=petro_ba90, remote=petro_ba90, display=Petro_Ba90>,
<QueryParameter full=nsa.petro_flux, name=petro_flux, short=petro_flux, remote=petro_flux, display=Petro_Flux>,
<QueryParameter full=nsa.petro_flux_ivar, name=petro_flux_ivar, short=petro_flux_ivar, remote=petro_flux_ivar, display=Petro_Flux_Ivar>,
<QueryParameter full=nsa.petro_phi50, name=petro_phi50, short=petro_phi50, remote=petro_phi50, display=Petro_Phi50>,
<QueryParameter full=nsa.petro_phi90, name=petro_phi90, short=petro_phi90, remote=petro_phi90, display=Petro_Phi90>,
<QueryParameter full=nsa.petro_th50, name=petro_th50, short=petro_th50, remote=petro_th50, display=Petro_Th50>,
<QueryParameter full=nsa.petro_th90, name=petro_th90, short=petro_th90, remote=petro_th90, display=Petro_Th90>,
<QueryParameter full=nsa.petro_theta, name=petro_theta, short=petro_theta, remote=petro_theta, display=Petro_Theta>,
<QueryParameter full=nsa.phistokes, name=phistokes, short=phistokes, remote=phistokes, display=Phistokes>,
<QueryParameter full=nsa.pid, name=pid, short=pid, remote=pid, display=Pid>,
<QueryParameter full=nsa.plate, name=plate, short=plate, remote=plate, display=Plate>,
<QueryParameter full=nsa.platequality, name=platequality, short=platequality, remote=platequality, display=Platequality>,
<QueryParameter full=nsa.plug_dec, name=plug_dec, short=plug_dec, remote=plug_dec, display=Plug_Dec>,
<QueryParameter full=nsa.plug_ra, name=plug_ra, short=plug_ra, remote=plug_ra, display=Plug_Ra>,
<QueryParameter full=nsa.profmean, name=profmean, short=profmean, remote=profmean, display=Profmean>,
<QueryParameter full=nsa.profmean_ivar, name=profmean_ivar, short=profmean_ivar, remote=profmean_ivar, display=Profmean_Ivar>,
<QueryParameter full=nsa.proftheta, name=proftheta, short=proftheta, remote=proftheta, display=Proftheta>,
<QueryParameter full=nsa.programname, name=programname, short=programname, remote=programname, display=Programname>,
<QueryParameter full=nsa.qstokes, name=qstokes, short=qstokes, remote=qstokes, display=Qstokes>,
<QueryParameter full=nsa.racat, name=racat, short=racat, remote=racat, display=Racat>,
<QueryParameter full=nsa.rerun, name=rerun, short=rerun, remote=rerun, display=Rerun>,
<QueryParameter full=nsa.run, name=run, short=run, remote=run, display=Run>,
<QueryParameter full=nsa.sersic_absmag, name=sersic_absmag, short=sersic_absmag, remote=sersic_absmag, display=Sersic_Absmag>,
<QueryParameter full=nsa.sersic_amivar, name=sersic_amivar, short=sersic_amivar, remote=sersic_amivar, display=Sersic_Amivar>,
<QueryParameter full=nsa.sersic_b1000, name=sersic_b1000, short=sersic_b1000, remote=sersic_b1000, display=Sersic_B1000>,
<QueryParameter full=nsa.sersic_b300, name=sersic_b300, short=sersic_b300, remote=sersic_b300, display=Sersic_B300>,
<QueryParameter full=nsa.sersic_flux, name=sersic_flux, short=sersic_flux, remote=sersic_flux, display=Sersic_Flux>,
<QueryParameter full=nsa.sersic_flux_ivar, name=sersic_flux_ivar, short=sersic_flux_ivar, remote=sersic_flux_ivar, display=Sersic_Flux_Ivar>,
<QueryParameter full=nsa.sersic_kcoeff, name=sersic_kcoeff, short=sersic_kcoeff, remote=sersic_kcoeff, display=Sersic_Kcoeff>,
<QueryParameter full=nsa.sersic_kcorrect, name=sersic_kcorrect, short=sersic_kcorrect, remote=sersic_kcorrect, display=Sersic_Kcorrect>,
<QueryParameter full=nsa.sersic_mass, name=sersic_mass, short=sersic_mass, remote=sersic_mass, display=Sersic_Mass>,
<QueryParameter full=nsa.sersic_mets, name=sersic_mets, short=sersic_mets, remote=sersic_mets, display=Sersic_Mets>,
<QueryParameter full=nsa.sersic_mtol, name=sersic_mtol, short=sersic_mtol, remote=sersic_mtol, display=Sersic_Mtol>,
<QueryParameter full=nsa.sersic_n, name=sersic_n, short=sersic_n, remote=sersic_n, display=Sersic_N>,
<QueryParameter full=nsa.sersic_nmgy, name=sersic_nmgy, short=sersic_nmgy, remote=sersic_nmgy, display=Sersic_Nmgy>,
<QueryParameter full=nsa.sersic_nmgy_ivar, name=sersic_nmgy_ivar, short=sersic_nmgy_ivar, remote=sersic_nmgy_ivar, display=Sersic_Nmgy_Ivar>,
<QueryParameter full=nsa.sersic_ok, name=sersic_ok, short=sersic_ok, remote=sersic_ok, display=Sersic_Ok>,
<QueryParameter full=nsa.sersic_phi, name=sersic_phi, short=sersic_phi, remote=sersic_phi, display=Sersic_Phi>,
<QueryParameter full=nsa.sersic_rnmgy, name=sersic_rnmgy, short=sersic_rnmgy, remote=sersic_rnmgy, display=Sersic_Rnmgy>,
<QueryParameter full=nsa.sersic_th50, name=sersic_th50, short=sersic_th50, remote=sersic_th50, display=Sersic_Th50>,
<QueryParameter full=nsa.size, name=size, short=size, remote=size, display=Size>,
<QueryParameter full=nsa.subdir, name=subdir, short=subdir, remote=subdir, display=Subdir>,
<QueryParameter full=nsa.survey, name=survey, short=survey, remote=survey, display=Survey>,
<QueryParameter full=nsa.tile, name=tile, short=tile, remote=tile, display=Tile>,
<QueryParameter full=nsa.ustokes, name=ustokes, short=ustokes, remote=ustokes, display=Ustokes>,
<QueryParameter full=nsa.xcen, name=xcen, short=xcen, remote=xcen, display=Xcen>,
<QueryParameter full=nsa.xpos, name=xpos, short=xpos, remote=xpos, display=Xpos>,
<QueryParameter full=nsa.ycen, name=ycen, short=ycen, remote=ycen, display=Ycen>,
<QueryParameter full=nsa.ypos, name=ypos, short=ypos, remote=ypos, display=Ypos>,
<QueryParameter full=nsa.zdist, name=zdist, short=zdist, remote=zdist, display=Zdist>,
<QueryParameter full=nsa.zsdssline, name=zsdssline, short=zsdssline, remote=zsdssline, display=Zsdssline>,
<QueryParameter full=nsa.zsrc, name=zsrc, short=zsrc, remote=zsrc, display=Zsrc>]
Multiple Search Criteria and Returning Additional Parameters¶
We can easily combine query filter conditions by constructing a boolean string using AND. Let’s search for galaxies with a redshift < 0.1 and log M\(_\star\) < 10. The NSA catalog contains the Sersic profile determination for stellar mass, which is the sersic_mass or sersic_logmass parameter of the ``nsa`` table, so our search parameter will be nsa.sersic_logmass.
Let’s also return the object RA and Dec as well using the return_params keyword. This accepts a list of string parameters. Object RA and Dec are included in the cube table so the parameter names are cube.ra and cube.dec.
[13]:
my_filter = 'nsa.z < 0.1 and nsa.sersic_logmass < 10'
q = Query(search_filter=my_filter, return_params=['cube.ra', 'cube.dec'])
r = q.run()
print(r)
print('Number of objects:', r.totalcount)
Marvin Results(query=nsa.z < 0.1 and nsa.sersic_logmass < 10, totalcount=1932, count=100, mode=remote)
Number of objects: 1932
This query return 1932 objects and now includes the RA, Dec, redshift and log Sersic stellar mass parameters.
[14]:
# print the first 10 rows
r.results[0:10]
[14]:
<ResultSet(set=1.0/194, index=0:10, count_in_set=10, total=1932)>
[ResultRow(mangaid='1-109073', plateifu='8078-12704', ra=42.1567757009, dec=-0.554292603774, sersic_logmass=9.89949219613813, z=0.0435882),
ResultRow(mangaid='1-109234', plateifu='9193-1901', ra=45.9974263376, dec=0.426382098814, sersic_logmass=9.71193033196987, z=0.0409309),
ResultRow(mangaid='1-109250', plateifu='8080-1902', ra=47.7852584272, dec=-1.05434100442, sersic_logmass=8.86787095265526, z=0.0383731),
ResultRow(mangaid='1-109270', plateifu='9193-12703', ra=46.6858583566, dec=-0.40880014755, sersic_logmass=9.13073892096969, z=0.0252266),
ResultRow(mangaid='1-109356', plateifu='8081-3703', ra=50.0738407878, dec=0.523632230733, sersic_logmass=9.48148132628646, z=0.0241782),
ResultRow(mangaid='1-113219', plateifu='7815-9102', ra=317.374745914, dec=10.0519434342, sersic_logmass=9.37199275559893, z=0.0408897),
ResultRow(mangaid='1-113322', plateifu='7972-12701', ra=315.533514754, dec=11.0751131018, sersic_logmass=9.95351566466319, z=0.0431628),
ResultRow(mangaid='1-113346', plateifu='7972-12702', ra=315.85387762, dec=10.8770313048, sersic_logmass=9.94489259080932, z=0.0431389),
ResultRow(mangaid='1-113375', plateifu='7815-9101', ra=316.639658795, dec=10.7512221884, sersic_logmass=9.82192731931789, z=0.028215),
ResultRow(mangaid='1-113375', plateifu='7972-12704', ra=316.639658795, dec=10.7512221884, sersic_logmass=9.82192731931789, z=0.028215)]
Radial Queries in Marvin¶
Cone searches can be performed with Marvin Queries using a special functional syntax in your SQL string. Cone searches can be performed using the special radial string function. The syntax for a cone search query is radial(RA, Dec, radius). Let’s search for all galaxies within 0.5 degrees centered on RA, Dec = 232.5447, 48.6902. The RA and Dec must be in decimal degrees and the radius is in units of degrees.
[15]:
# build the radial filter condition
my_filter = 'radial(232.5447, 48.6902, 0.5)'
q = Query(search_filter=my_filter)
r = q.run()
print(r)
print(r.results)
Marvin Results(query=radial(232.5447, 48.6902, 0.5), totalcount=2, count=2, mode=remote)
<ResultSet(set=1.0/1, index=0:2, count_in_set=2, total=2)>
[ResultRow(mangaid='1-209232', plateifu='8485-1901', ra=232.544703894, dec=48.6902009334),
ResultRow(mangaid='1-209266', plateifu='8485-9101', ra=233.107502765, dec=48.8332849239)]
Queries using DAPall parameters.¶
MaNGA provides derived analysis properties in its dapall summary file. Marvin allows for queries on any of the parameters in the file. The table name for these parameters is dapall. Let’s find all galaxies that have a total measure star-formation rate > 5 M\(_\odot\)/year. The total SFR parameter in the DAPall table is sfr_tot.
[17]:
my_filter = 'dapall.sfr_tot > 5'
q = Query(search_filter=my_filter)
r = q.run()
print(r)
print(r.results)
Marvin Results(query=dapall.sfr_tot > 5, totalcount=6, count=6, mode=remote)
<ResultSet(set=1.0/1, index=0:6, count_in_set=6, total=6)>
[ResultRow(mangaid='1-24092', plateifu='7991-1901', sfr_tot=5.5633, bintype_name='VOR10', template_name='GAU-MILESHC'),
ResultRow(mangaid='1-24092', plateifu='7991-1901', sfr_tot=5.69988, bintype_name='HYB10', template_name='GAU-MILESHC'),
ResultRow(mangaid='1-37863', plateifu='9193-12704', sfr_tot=6.457, bintype_name='VOR10', template_name='GAU-MILESHC'),
ResultRow(mangaid='1-37863', plateifu='9193-12704', sfr_tot=6.82693, bintype_name='HYB10', template_name='GAU-MILESHC'),
ResultRow(mangaid='1-43214', plateifu='8135-1902', sfr_tot=26.4648, bintype_name='VOR10', template_name='GAU-MILESHC'),
ResultRow(mangaid='1-43214', plateifu='8135-1902', sfr_tot=27.4152, bintype_name='HYB10', template_name='GAU-MILESHC')]
The query returns 6 results, but looking at the plateifu, we see there are only 3 unique targets. This is because the DAPall file provides measurements for multiple bintypes and by default will return entries for all bintypes. We can select those out using the bintype.name parameter. Let’s filter on only the HYB10 bintype.
[18]:
my_filter = 'dapall.sfr_tot > 5 and bintype.name==HYB10'
q = Query(search_filter=my_filter)
r = q.run()
print(r)
print(r.results)
Marvin Results(query=dapall.sfr_tot > 5 and bintype.name==HYB10, totalcount=3, count=3, mode=remote)
<ResultSet(set=1.0/1, index=0:3, count_in_set=3, total=3)>
[ResultRow(mangaid='1-24092', plateifu='7991-1901', sfr_tot=5.69988, bintype_name='HYB10', template_name='GAU-MILESHC'),
ResultRow(mangaid='1-37863', plateifu='9193-12704', sfr_tot=6.82693, bintype_name='HYB10', template_name='GAU-MILESHC'),
ResultRow(mangaid='1-43214', plateifu='8135-1902', sfr_tot=27.4152, bintype_name='HYB10', template_name='GAU-MILESHC')]
Query on Quality and Target Flags¶
Marvin includes the ability to perform queries using quality or target flag information. These work using the special quality and targets keyword arguments. These keywords accept a list of flag maskbit labels provided by the Maskbit Datamodel. These keywords are inclusive, meaning they will only filter on objects satisfying those labels.
Searching by Target Flags¶
Let’s find all galaxies that are in the MaNGA MAIN target selection sample. Targets in the MAIN sample are a part of the PRIMARY, SECONDARY and COLOR-ENHANCED samples. These are the primary, secondary, and color-enhanced flag labels. The targets keywords accepts all labels from the MANGA_TARGET1, MANGA_TARGET2, or MANGA_TARGET3 maskbit schema.
[19]:
# create the targets list of labels
targets = ['primary', 'secondary', 'color-enhanced']
q = Query(targets=targets)
r = q.run()
print(r)
print('There are {0} galaxies in the main sample'.format(r.totalcount))
print(r.results[0:5])
Marvin Results(query=None, totalcount=2836, count=100, mode=remote)
There are 2836 galaxies in the main sample
<ResultSet(set=1.0/568, index=0:5, count_in_set=5, total=2836)>
[ResultRow(mangaid='1-109056', plateifu='8077-6103', manga_target1=2080),
ResultRow(mangaid='1-109073', plateifu='8078-12704', manga_target1=2336),
ResultRow(mangaid='1-109081', plateifu='8077-12705', manga_target1=4112),
ResultRow(mangaid='1-109112', plateifu='8078-1901', manga_target1=1040),
ResultRow(mangaid='1-109152', plateifu='8154-12702', manga_target1=4096)]
The targets keyword is equivalent to the cube.manga_targetX search parameter, where X is 1, 2, or 3. The bits for the primary, secondary, and color-enhanced samples are 10, 11, and 12, respectively. These combine into the value 7168. The above query is equivalent to the filter condition cube.manga_target1 & 7168
[20]:
value = 1<<10 | 1<<11 | 1<<12
my_filter = 'cube.manga_target1 & {0}'.format(value)
q = Query(search_filter=my_filter)
r = q.run()
print(r)
Marvin Results(query=cube.manga_target1 & 7168, totalcount=4498, count=100, mode=remote)
Let’s search only for galaxies that are Milky Way Analogs or Dwarfs ancillary targets.
[21]:
targets = ['mwa', 'dwarf']
q = Query(targets=targets)
r = q.run()
print(r)
print('There are {0} galaxies from the Milky Way Analogs and Dwarfs ancillary target catalogs'.format(r.totalcount))
print(r.results)
Marvin Results(query=None, totalcount=26, count=26, mode=remote)
There are 26 galaxies from the Milky Way Analogs and Dwarfs ancillary target catalogs
<ResultSet(set=1.0/1, index=0:26, count_in_set=26, total=26)>
[ResultRow(mangaid='1-121994', plateifu='9485-9102', manga_target3=16384),
ResultRow(mangaid='1-124604', plateifu='8439-6103', manga_target3=8192),
ResultRow(mangaid='1-135491', plateifu='9869-6101', manga_target3=8192),
ResultRow(mangaid='1-146067', plateifu='8937-6102', manga_target3=16384),
ResultRow(mangaid='1-187821', plateifu='8996-6104', manga_target3=16384),
ResultRow(mangaid='1-198869', plateifu='8551-6101', manga_target3=16384),
ResultRow(mangaid='1-199488', plateifu='9038-6103', manga_target3=16384),
ResultRow(mangaid='1-218519', plateifu='8942-12703', manga_target3=16384),
ResultRow(mangaid='1-234408', plateifu='8319-6102', manga_target3=8192),
ResultRow(mangaid='1-23818', plateifu='8611-6104', manga_target3=16384),
ResultRow(mangaid='1-276557', plateifu='8993-6101', manga_target3=16384),
ResultRow(mangaid='1-295506', plateifu='9049-6101', manga_target3=16384),
ResultRow(mangaid='1-321962', plateifu='8552-6101', manga_target3=16384),
ResultRow(mangaid='1-338566', plateifu='8566-6103', manga_target3=16384),
ResultRow(mangaid='1-351538', plateifu='8567-6101', manga_target3=8192),
ResultRow(mangaid='1-384388', plateifu='9494-6102', manga_target3=16384),
ResultRow(mangaid='1-384394', plateifu='9494-6103', manga_target3=16384),
ResultRow(mangaid='1-384930', plateifu='9493-12702', manga_target3=16384),
ResultRow(mangaid='1-385149', plateifu='9502-6102', manga_target3=16384),
ResultRow(mangaid='1-386657', plateifu='8987-6101', manga_target3=16384),
ResultRow(mangaid='1-386800', plateifu='8987-6103', manga_target3=16384),
ResultRow(mangaid='1-415958', plateifu='8985-6101', manga_target3=16384),
ResultRow(mangaid='1-457843', plateifu='8982-12702', manga_target3=16384),
ResultRow(mangaid='43-130', plateifu='9507-6104', manga_target3=16384),
ResultRow(mangaid='43-22', plateifu='8713-6101', manga_target3=16384),
ResultRow(mangaid='43-47', plateifu='8725-9101', manga_target3=16384)]
Searching by Quality Flags¶
The quality accepts all labels from the MANGA_DRPQUAL and MANGA_DAPQUAL maskbit schema. Let’s find all galaxies that suffered from bad flux calibration. This is the flag BADFLUX (bit 8) from the MANGA_DRPQUAL maskbit schema.
[22]:
quality = ['BADFLUX']
q = Query(quality=quality)
r = q.run()
print(r)
print('There are {0} galaxies with bad flux calibration'.format(r.totalcount))
print(r.results[0:10])
Marvin Results(query=None, totalcount=82, count=82, mode=remote)
There are 82 galaxies with bad flux calibration
<ResultSet(set=1.0/9, index=0:10, count_in_set=10, total=82)>
[ResultRow(mangaid='1-109493', plateifu='8156-9101', quality=1073742144),
ResultRow(mangaid='1-113273', plateifu='7972-1902', quality=1073742144),
ResultRow(mangaid='1-120967', plateifu='8144-12704', quality=1073742144),
ResultRow(mangaid='1-121035', plateifu='8144-1901', quality=1073742144),
ResultRow(mangaid='1-136304', plateifu='8606-1902', quality=1073742144),
ResultRow(mangaid='1-138157', plateifu='8252-9102', quality=1073742144),
ResultRow(mangaid='1-149170', plateifu='8997-3701', quality=1073742144),
ResultRow(mangaid='1-152527', plateifu='8144-6101', quality=1073742144),
ResultRow(mangaid='1-152769', plateifu='8936-3704', quality=1073742144),
ResultRow(mangaid='1-174629', plateifu='8947-3703', quality=1073742080)]
The quality keyword is equivalent to the search parameters cube.quality for DRP flags or the file.quality for DAP flags. The above query is equivalent to cube.quality & 256. You can also perform a NOT bitmask selection using the ~ symbol. To perform a NOT selection we can only use the cube.quality parameter. Let’s select all galaxies that do not have bad flux calibration.
[23]:
# the above query as a filter condition
q = Query(search_filter='cube.quality & 256')
r = q.run()
print('Objects with bad flux calibration:', r.totalcount)
# objects with bad quality other than bad flux calibration
q = Query(search_filter='cube.quality & ~256')
r = q.run()
print('Bad objects with no bad flux calibration:', r.totalcount)
Objects with bad flux calibration: 82
Bad objects with no bad flux calibration: 604
To find exactly objects with good quality and no bad flags set, use cube.quality == 0.
[24]:
q = Query(search_filter='cube.quality == 0')
r = q.run()
print(r)
print('Objects with good quality:', r.totalcount)
Marvin Results(query=cube.quality == 0, totalcount=4253, count=100, mode=remote)
Objects with good quality: 4253
Useful Resources¶
Check out these pages on the Marvin Docs site for more information querying with Marvin.
[ ]: