Retrieving Your Results¶
By default, Marvin will paginate large query results to limit strains on bandwidth. For results with over 1000 objects,
Marvin automatically paginates results in groups of Query.limit
, which defaults to 100. This page describes various methods
of retrieving your query results.
Getting All the Results¶
If you’re feeling rebellious, we provide a method of retrieving all the results in a single request, with getAll
:
r.count, r.totalcount
(100, 1282)
r.getAll()
Returned all 1282 results
r.results
<ResultSet(set=1/1, index=0:1281, count_in_set=1282, total=1282)>
[ResultRow(mangaid=u'1-109394', plate=8082, plateifu=u'8082-9102', ifu_name=u'9102', elpetro_absmag_g_r=1.26038932800293, z=0.0361073),
ResultRow(mangaid=u'1-113208', plate=8618, plateifu=u'8618-3701', ifu_name=u'3701', elpetro_absmag_g_r=1.48788070678711, z=0.0699044),
ResultRow(mangaid=u'1-113219', plate=7815, plateifu=u'7815-9102', ifu_name=u'9102', elpetro_absmag_g_r=0.543312072753906, z=0.0408897),
ResultRow(mangaid=u'1-113375', plate=7815, plateifu=u'7815-9101', ifu_name=u'9101', elpetro_absmag_g_r=0.757579803466797, z=0.028215),
ResultRow(mangaid=u'1-113379', plate=7815, plateifu=u'7815-6101', ifu_name=u'6101', elpetro_absmag_g_r=1.09770011901855, z=0.0171611),
...]
This method has a built-in limit. It will not return all the results for results with over 500,000 rows and/or 50 columns of data. You may
override this limit with the force
keyword argument.:
r.getAll(force=True)
Depending on the number of results, the amount of data being returned, and the bandwidth of your internet connection,
the getAll
method may be very slow, or simply not work entirely. Therefore, we also provide a number of methods to page through your
results and access them in piecemeal. The remaining sections describe these alternative methods.
Extending the Set¶
You can extend your current set of results with the next set using extendSet
. This retrieves the next page of results and
adds it to your current page:
# count in current set
r.count
100
# the ResultSet
r.results
<ResultSet(set=1/13, index=0:100, count_in_set=100, total=1282)>
[ResultRow(mangaid=u'1-109394', plate=8082, plateifu=u'8082-9102', ifu_name=u'9102', elpetro_absmag_g_r=1.26038932800293, z=0.0361073),
ResultRow(mangaid=u'1-113208', plate=8618, plateifu=u'8618-3701', ifu_name=u'3701', elpetro_absmag_g_r=1.48788070678711, z=0.0699044),
ResultRow(mangaid=u'1-113219', plate=7815, plateifu=u'7815-9102', ifu_name=u'9102', elpetro_absmag_g_r=0.543312072753906, z=0.0408897),
ResultRow(mangaid=u'1-113375', plate=7815, plateifu=u'7815-9101', ifu_name=u'9101', elpetro_absmag_g_r=0.757579803466797, z=0.028215),
ResultRow(mangaid=u'1-113379', plate=7815, plateifu=u'7815-6101', ifu_name=u'6101', elpetro_absmag_g_r=1.09770011901855, z=0.0171611),
ResultRow(mangaid=u'1-113403', plate=7815, plateifu=u'7815-12703', ifu_name=u'12703', elpetro_absmag_g_r=0.745466232299805, z=0.0715126
...]
# extend the set with the next chunk of data
r.extendSet()
INFO: Retrieving next 100, from 100 to 200
r.count
200
r.results
<ResultSet(set=1/7, index=0:200, count_in_set=200, total=1282)>
[ResultRow(mangaid=u'1-109394', plate=8082, plateifu=u'8082-9102', ifu_name=u'9102', elpetro_absmag_g_r=1.26038932800293, z=0.0361073),
ResultRow(mangaid=u'1-113208', plate=8618, plateifu=u'8618-3701', ifu_name=u'3701', elpetro_absmag_g_r=1.48788070678711, z=0.0699044),
ResultRow(mangaid=u'1-113219', plate=7815, plateifu=u'7815-9102', ifu_name=u'9102', elpetro_absmag_g_r=0.543312072753906, z=0.0408897),
ResultRow(mangaid=u'1-113375', plate=7815, plateifu=u'7815-9101', ifu_name=u'9101', elpetro_absmag_g_r=0.757579803466797, z=0.028215),
ResultRow(mangaid=u'1-113379', plate=7815, plateifu=u'7815-6101', ifu_name=u'6101', elpetro_absmag_g_r=1.09770011901855, z=0.0171611),
ResultRow(mangaid=u'1-113403', plate=7815, plateifu=u'7815-12703', ifu_name=u'12703', elpetro_absmag_g_r=0.745466232299805, z=0.0715126),
ResultRow(mangaid=u'1-113418', plate=7815, plateifu=u'7815-12704', ifu_name=u'12704', elpetro_absmag_g_r=1.44098854064941, z=0.0430806),
...]
You can grow your set by a larger number than the default with the chunk
keyword argument.:
# extend your set of results by
r.extendSet(chunk=500)
Loop over the Data¶
You can attempt to loop over all the results, automatically calling extendSet
, to grow your results to the total count. loop
also accepts a chunk
keyword argument to vary the amount of rows returned in each iteration.
r.loop()
Retrieving next 100, from 100 to 200
Retrieving next 100, from 200 to 300
Retrieving next 100, from 300 to 400
Retrieving next 100, from 400 to 500
Retrieving next 100, from 500 to 600
Retrieving next 100, from 600 to 700
Retrieving next 100, from 700 to 800
Retrieving next 100, from 800 to 900
Retrieving next 100, from 900 to 1000
Retrieving next 100, from 1000 to 1100
Retrieving next 100, from 1100 to 1200
WARNING: You have reached the end.
Retrieving next 100, from 1200 to 1282
r.count
1282
Both extendSet
and loop
will grow the existing ResultSet
to encompass all of the results. Use these if you want to get all the results
locally. The following methods work only on specific pages at a time, and replace your existing ResultSet
.
Get Next/Previous Chunks in List¶
You can view, and grab, the next or previous chunk with
r.getNext()
r.getPrevious()
You can also specify a chunk value
# get next 5 entries
r.getNext(chunk=5)
Retrieving next 5, from 100 to 105
<ResultSet(set=21/257, index=100:105, count_in_set=5, total=1282)>
[ResultRow(mangaid=u'1-135548', plate=8601, plateifu=u'8601-12702', ifu_name=u'12702', elpetro_absmag_g_r=1.05030250549316, z=0.030559),
ResultRow(mangaid=u'1-135568', plate=8601, plateifu=u'8601-12701', ifu_name=u'12701', elpetro_absmag_g_r=0.790615081787109, z=0.0938565),
ResultRow(mangaid=u'1-135641', plate=8588, plateifu=u'8588-12704', ifu_name=u'12704', elpetro_absmag_g_r=1.44169998168945, z=0.030363),
ResultRow(mangaid=u'1-135657', plate=8588, plateifu=u'8588-1901', ifu_name=u'1901', elpetro_absmag_g_r=1.22106170654297, z=0.0364618),
ResultRow(mangaid=u'1-135679', plate=8588, plateifu=u'8588-6103', ifu_name=u'6103', elpetro_absmag_g_r=1.4596061706543, z=0.0331057)]
# get previous 5 entries
r.getPrevious(chunk=5)
Retrieving previous 5, from 95 to 100
<ResultSet(set=20/257, index=95:100, count_in_set=5, total=1282)>
[ResultRow(mangaid=u'1-135512', plate=8601, plateifu=u'8601-6102', ifu_name=u'6102', elpetro_absmag_g_r=0.778741836547852, z=0.0279629),
ResultRow(mangaid=u'1-135516', plate=8550, plateifu=u'8550-6104', ifu_name=u'6104', elpetro_absmag_g_r=1.33112716674805, z=0.0314747),
ResultRow(mangaid=u'1-135517', plate=8588, plateifu=u'8588-6101', ifu_name=u'6101', elpetro_absmag_g_r=1.17428970336914, z=0.0317611),
ResultRow(mangaid=u'1-135530', plate=8550, plateifu=u'8550-9101', ifu_name=u'9101', elpetro_absmag_g_r=1.7724609375, z=0.0283296),
ResultRow(mangaid=u'1-135545', plate=8601, plateifu=u'8601-6103', ifu_name=u'6103', elpetro_absmag_g_r=1.43307685852051, z=0.0301334)]
Get Subset¶
To retrieve a subset of the results, use r.getSubset
. getSubset works by specifying the starting index to grab from and a limit
on the number to grab (default is 10). Having the returntype specified will also generate the corresponding Marvin Tools for the new
subset
# Get a subet of objects starting at index 300 (note the chunk is now 5, due to the above code example)
r.getSubset(start=300)
<ResultSet(set=61/257, index=300:305, count_in_set=5, total=1282)>
[ResultRow(mangaid=u'1-211227', plate=8603, plateifu=u'8603-12702', ifu_name=u'12702', elpetro_absmag_g_r=1.62772369384766, z=0.0276667),
ResultRow(mangaid=u'1-211239', plate=8550, plateifu=u'8550-6102', ifu_name=u'6102', elpetro_absmag_g_r=2.10557651519775, z=0.0265494),
ResultRow(mangaid=u'1-211311', plate=8550, plateifu=u'8550-3704', ifu_name=u'3704', elpetro_absmag_g_r=1.36248779296875, z=0.0298414),
ResultRow(mangaid=u'1-216520', plate=8440, plateifu=u'8440-3701', ifu_name=u'3701', elpetro_absmag_g_r=1.60188102722168, z=0.0239988),
ResultRow(mangaid=u'1-216540', plate=8440, plateifu=u'8440-12701', ifu_name=u'12701', elpetro_absmag_g_r=1.17086791992188, z=0.0289061)]
# Get a subset of 10 objects starting at index 500
r.getSubset(start=500, limit=10)
<ResultSet(set=51/129, index=500:510, count_in_set=10, total=1282)>
[ResultRow(mangaid=u'1-256465', plate=8451, plateifu=u'8451-6104', ifu_name=u'6104', elpetro_absmag_g_r=1.55233573913574, z=0.0574997),
ResultRow(mangaid=u'1-256496', plate=8258, plateifu=u'8258-3704', ifu_name=u'3704', elpetro_absmag_g_r=0.784034729003906, z=0.0584715),
ResultRow(mangaid=u'1-256506', plate=8258, plateifu=u'8258-3703', ifu_name=u'3703', elpetro_absmag_g_r=1.70734596252441, z=0.0587373),
ResultRow(mangaid=u'1-256546', plate=8258, plateifu=u'8258-12702', ifu_name=u'12702', elpetro_absmag_g_r=0.911626815795898, z=0.0212478),
ResultRow(mangaid=u'1-256574', plate=8258, plateifu=u'8258-12703', ifu_name=u'12703', elpetro_absmag_g_r=1.10363006591797, z=0.0660807),
ResultRow(mangaid=u'1-256647', plate=8466, plateifu=u'8466-6102', ifu_name=u'6102', elpetro_absmag_g_r=1.71620178222656, z=0.0487626),
ResultRow(mangaid=u'1-256819', plate=8466, plateifu=u'8466-1902', ifu_name=u'1902', elpetro_absmag_g_r=1.69637107849121, z=0.0662541),
ResultRow(mangaid=u'1-256860', plate=8466, plateifu=u'8466-9101', ifu_name=u'9101', elpetro_absmag_g_r=1.00818252563477, z=0.0245857),
ResultRow(mangaid=u'1-25688', plate=7990, plateifu=u'7990-6103', ifu_name=u'6103', elpetro_absmag_g_r=1.4294376373291, z=0.0292359),
ResultRow(mangaid=u'1-257100', plate=8466, plateifu=u'8466-12705', ifu_name=u'12705', elpetro_absmag_g_r=1.02963256835938, z=0.045258)]
Downloading Results¶
Download the results of your query. This downloads the MaNGA FITS file data products for all the objects in your current ResultSet
.
The downloaded object (FITS file) is determined by the returntype parameter, which defaults to cube if not specified.
# downloads all the data using sdss_acccess
r.download()
download
also accepts a limit
keyword, to limit the number of downloaded objects:
r.download(limit=5)