Marvin’s Change Log¶
[2.8.0] - 2022/08/17¶
Drop support for py <3.7. Min. version is py3.8.
Updating flask web dependencies: Flask > 2, Jinja2 > 3, werkzeug > 2.0, <2.1
Removing packaged Flask-JSGlue, replacing with custom 0.3.1 version with correct use of
Pinning miniumum numpy version to 1.20, removing use of
distutilsfor version checking
Updates test suite to run only on DR17, and on CI on Github Actions. Removes travis yaml.
[2.7.4] - 2022/07/27¶
Last tag to support python versions >2.7 up to <3.8
Additional docs updates
[2.7.3] - 2022/07/12¶
Small tweaks to documentation links and tutorial exercise notebooks
Pinning markupsafe dependency to < 2.1
[2.7.2] - 2021/12/07¶
[2.7.1] - 2021/12/05¶
Bug fix in Query datamodel. Removed older MPLs from the datamodel as they are no longer in the database.
[2.7.0] - 2021/12/05¶
Adds support for DR17, aliases to MPL-11
Fixing bug in core Marvin Tools
closemethod, for use as contextmanager
Fixing bug in web galaxy template when no nsa data available; so pages do not crash
Added new explore page in the web to upload a target list and display DAP maps
Adds basic VAC availablity to galaxy web page
Updates the Marvin Query
Results.toJsonmethod to return a more flexible JSON format
Improves performance of web query table
Adds export table button option to web query table
Improves web pages with various tooltips and other small improvements
Fixes bug in
Adds support for DR17 DAPTYPES SPX and VOR10
Removes web and API support for MPL-8, MPL-9, and MPL-10
[2.6.1] - 2021/11/18¶
Issue #680 - Basic context manager functionality for
marvin.tools.core.MarvinToolsClass, and relevant documentation for
Maps. Users can automate closing file handlers when in
filemode, by using
withsyntax, alleviating some issues with looping through lists of galaxies.
marvin.tools.core.MarvinToolsClass.closemethod is a self-contained way of closing the underlying data file. It is now called by
marvin.tools.core.MarvinToolsClass.source_is_fits_hdulist_fileevaluates whether the object is accessing a FITS HDUList.
Updated the manga database connection info for JHU
[2.6.0] - 2021/03/12¶
Adds - support for MaNGA data release MPL-11
Adds - updates to the Marvin documentation on citations
Fixes - Bug with remote access for removed
emline_tplsigmaspaxel property. Removed from MPL-10, MPL-11 datamodels.
[2.5.4] - 2021/03/5¶
** Changes** - pinned
flask-jwt-extendedpackage version to
[2.5.3] - 2020/16/1¶
Changes - static web images to webp format for better compression and faster load times.
[2.5.2] - 2020/10/1¶
Restructuring to simplify setup, dependencies, and tests, for code health.
Adds - New
setup.cfgto simplify installation and dependency requirements
Adds - New dependencies
Changes - Migrates to
sdsstoolsfor internal logging and config management.
Changes - Removes all external submodules, which have been converted into pypi releases.
Changes - Moves tests outside the python package
[2.5.1] - 2020/09/25¶
Fixes #711 - galaxy page bug when NSA dictionary not available prevents flag modals from appearing
[2.5.0] - 2020/09/04¶
Adds - support for MaNGA data release MPL-10
Removes - web / API support for MPLs 4, 5, 6
Removes - SPX and VOR10 binning schemese in MPLs 7-10, and DR15-16.
[2.4.1] - 2020/08/05¶
Adds - support for using the marvin mirror at magrathea.sdss.org, hosted at Johns Hopkins
Adds - new
use_mirrorcustom config option to automatically set marvin to use the mirror
[2.4.0] - 2020/07/19¶
Adds - caching web routes on Plate, Galaxy pages
Adds - new
Cachable Queryclass. Implemented in most marvin database calls.
Adds - most Marvin Tools now use query caching for database calls
Adds - a new
CacheMixInthat adds optional caching to classes utilizing db calls
Adds - a new
dogpile.cachebackend; with new regions for
Adds - new utility functions for DRPall, and DAPall files,
Adds - adds visual morphology VAC for DR16+
Fixes - Issue #695 - better web spectra/map error messages;
Changes - tree and sdss_access submodules to latest 3.0+ and 1.0+ versions
Changes - path changes to accommodate new tree and sdss_access
[2.3.6] - 2020/04/14¶
[2.3.5] - 2019/12/09¶
Support for MPL-9 MaNGA data
Issue #648 - Option to toggle common lines from the DAP in web spectrum viewer
Issue #112 - Option to toggle between rest-frame and observed wavelength in web spectrum viewer
Issue #679 - Option to toggle on DONOTUSE pixel region masks in web spectrum viewer
master-fifo file to uwsgi base config ini file
adds an alert to authenticate when no static image detected
updated datamodels to include MPL-9 changes
turned off API rate limiting globally
turned back on old spaxel query method
switched random image web page to use improved
changing galaxy page static image grab to use
[2.3.4] - 2019/12/09¶
Added explicit python path in uwsgi prod.ini file
Bug in session variable controlling DAP bintemps
[2.3.3] - 2019/12/09¶
Support for DR16. Updated datamodels and docs.
Added Galaxy Zoo VAC
Added GEMA VAC
Added Firefly VAC
Added Visual Morphology VAC
Merged #678, richer support for VACs
Merged #652, richer support for Windows
Merged #677, updated documentation on issues with fuzzy string matching
Merged #687, fixes map arithmetic with reflexive operators
all yaml.load uses new Loader to accommodate old and new yaml spec;
updated Runtime Issues documentation to include section on numpy.ufunc binary warnings
added to reprs for some db ModelClasses
sdss_access.sync.RsyncAccesshas been converted to
sdss_access.sync.Accessto allow for Linux/Windows switching
updates citation to Marvin paper
DAP Maps label on galaxy page now contains a pop up of links for more info
Issue #658 - modelcube datamodel units missing angstrom
Issue #655 - added yaml loader to remove yaml warnings for 5.1 spec
Fixed bug when server tries to access NSA on cube when none exists and triggers remote authentication issue
Issue #664 - Fixed link to MaNGA’s Getting Started in docs.
Issue #686 - added reflexive operators to EnhancedMap
Issue #689 - added link to CSV file of Firefly stellar mass measurements needed for the Spatially-Resolved Mass-Metallicity Relation Tutorial
Issue #682 - adds info links next to DAP Maps selection on galaxy page
Issue #581 - makes MapSpecView button a session variable
Issuu #208 - makes map selections session variable
Issue #171 - fix y-min to 0 for spectrum web view
[2.3.2] - 2019/02/27¶
Support for MPL-8
[2.3.0] - 2018/12/03¶
Syntax on the inputs to the
Resultstools has been changed.
DAP spaxel queries have been disabled due to performance issues. We expect to bring them back soon. Metadata queries (those querying the NSA or DAPall tables) are still available.
getSpaxelnow only loads the quantities from the parent object (that means that, for example,
Mapsproperties by default). Additional quantities can be loaded using
getSpaxelaccepted arguments have been changed to
modelcube. The formerly accepted arguments (
model(s)) now raise a deprecation error.
Added cheatsheet to docs.
New Web authentication using Flask-Login
New API authentication using Flask-JWT-Extended
Adds MPL-7 / DR15 datamodel
New config.access attribute indicating public or collab access
New config.login method to handle token-based login
New marvin.yml config file for customization of configuration options
Adds User table into the history schema of mangadb. Tracks user logins.
Maphas a new method
getSpaxelto retrieve an
Spaxelusing the parent
New configuration option in
default_release, to set the release to be used when Marvin gets imported (#463).
Applying a NumPy ufunc (except
NotImplementedErrorbecause ivar propagation is not implemented yet.
Marvin ImageTool to load optical images using the MMA (#22)
MMAMixInfor providing multi-modal data access
Querytool now accepts a new
qualitykeyword argument which enables querying on target or quality maskbit labels. (#485)
Added a new API route for streaming large query results. This uses a generator to stream large results back to the client to minimize memory usage and bulk responses.
Integrated datamodel plotting params into actual datamodel structures
Moved netrc checks into the Brain
Expanded sasurl into public and collab urls
Changes personal emails to sdss helpdesk email in web
Added rawsql and return_params columns to history.query table in mangadb
Extra keyword arguments passed to
Spectrum.plotare now forwarded to
Maps) can now be accessed from the
ax_setup()function is now hidden.
extern/and added a readme file for the external packages (#468).
plotnow only masks part of the spectrum that have the
DONOTUSEmaskbit set (#455).
pixmaskis now available for all quantities (except
AnalysisProprty). The property
maskednow uses the bit
DONOTUSEto determine what values must be masked out (#462).
Raises error when applying
stellar_sigmaMPL-6 maps. Applies correction to stellar_sigma and emline_sigma for web maps with added ‘Corrected’ title (#478)
Removes targeting bits from
The name of the channel is now shown in the
Replaced inconsistent parameter
MarvinError now accepts an optional
ignore_gitkeyword to locally turn off the git addition to the message
Resultsnow calls the streaming API route instead.
ModelCubeare instantiated from file, gunzip the file to a temporary location to speed up subsequent access (#525).
Convert MMA warnings to debug messages (#580).
Issue #421 - query returning spaxel props returns wrong total count
Bugfix - Python 3 xrange syntax bug in buildImageDict
Bin._create_spaxelsinstantiating spaxels with the wrong
(i,j)values for the bin. The
(i, j)values from the
binidmap were being swapped twice before sending them to
A bug in the calculation of the deredden inverse variance in a
Issue with setting drpall path on initial import/set of release before tree has been planted
Issue #456 - spectrum web display shows incorrect RA, Dec
Issue #422 - ensuring config auto checks access to netrc
Issue #423 - adding marvin.yml documentation
Issue #431 - adding login documentation
Issue #151 - adding web spectrum tooltips
Issue #548 - failed to retrieve ModelCube extension in remote mode
Fixed typo by in method name
Spectrum.derredden -> Spectrum.deredden.
Fixed #305 - adding ivar propogation for np.log10(Map)
A bug when explicitly returning default parameters in a query (#484)
Fixed #510 - fixes incorrect conversion to sky coordinates in map plotting.
Fixed #563 - fail retrieving Query datamodels in Python 3.6+.
Fixes bug with sasurl not properly being set to api.sdss.org on initial import
Incorrect setting of the default bintype to download from web (#531).
Incorrect mismatch warning between MPL-7 and DR15 (#495).
Incorrect handling of maskbits when the mask does not contain any of the bits in the list (#507).
Fixed #534 - flipped axes in NSA scatterplot when plotting absmag colors
Fixed #559 - bug in check_marvin when marvindb is None
Fixed #579 - bug in MMA with marvindb preventing files from opening
Fixed #575 - cannot access maps due to bug in login and authentication in Interaction class
Fixed #539 - print downloadList target directory
Fixed #566 - made error message for web query with non-unique parameters name more specific
marvin.tools.coreand split the mixins into
Moved pieces of MarvinToolsClass into
Queryto remove local query dependencies from remote mode usage.
[2.2.5] - 2018/04/26¶
Galaxy Web page spaxel loading to be robust when no modelspaxels are present in the database.
[2.2.3] - 2018/03/20¶
Added tests for
emline_gflux_ha_6564and fixed values in galaxy_test_data.
Issue #374: Cube units do not persist under axis reordering.
Fixed some problems with test_spaxel tests.
Issue #382: Is fuzzywuzzy too fuzzy?
Fixed an issue with Astropy 3 in
Fixed some issues with query results tests
The banner that showed up in Safari has been removed since most versions should now work properly.
[2.2.2] - 2018/02/25¶
MPL-6 issue with all H-alpha extensions mapped to NII instead. Indexing issue in MPL-6 datamodel.
MPL-6 issue with elliptical coordinate extensions; missing R/Reff channel in MPL-6 datamodel.
[2.2.0] - 2018/01/12¶
Maskbitclass for easy conversion between mask values, bits, and labels.
Better BPT documentation, in particular in the
Modifying the plotsection.
A hack function
marvin.utils.plot.utils.bind_to_figure()that replicate the contents of a matplotlib axes in another figure.
New scatter and histogram plotting utility functions
Integrated scatter and histogram plotting into query Results
New methods for easier query Results handling
New Pythonic DRP, DAP, and Query DataModels
Access to DAPall data
marvin.utils.dap.bpt.bpt_kewley06()now also return a list of axes. Each axes contains a method pointing to the
marvin.utils.plot.utils.bind_to_figure()function, for easily transfer the axes to a new figure.
All Cubes, Maps, and Modelcubes use Astropy Quantities
Refactored to the Bin class
Bin and Spaxel are now subclassed from SpaxelBase
[2.1.4] - 2017/08/02¶
Added new query_params object, for easier navigation of available query parameters. Added new tests.
Added a new guided query builder using Jquery Query Builder to the Search page
Added a View Galaxies link on the web results to view postage stamps of the galaxies in the results
Added Route Rate Limiting. Adopts a limit of 200/min for all api routes and 60/minute for query api calls and web searches
Changed call signature for :meth:
valuearg because unused, added
Noneas default value
None), and re-ordered args and kwargs).
Changed call signature for :meth:
valuearg because unused).
Changed the Marvin web search page to use the new query_params and parameter grouping. Removed the autocomplete input box.
Updated the documentation on query and query_params.
Modified Guided Search operator options to remove options that could not be parsed by SQLA boolean_search
Refactored the web settings, route registration, extensions to enable extensibility
Issue #282: Improvements to “Go to CAS” link. Changed to Go To SkyServer and updated link to public up-to-date link
Issue #102: problem with urllib package when attempting to retrieve the Marvin URLMap
Issue #93: safari browser does not play well with marvin
Issue #155: Contrails in Web Map
Issue #174: sdss_access may not be completely python 3 compatible
Issue #196: Bin not loading from local sas
Issue #207: Get Maps in MapSpecView of Galaxy page sometimes fails to return selected maps
Issue #210: pip upgrade may not install new things as fresh install
Issue #209: marvin version from pip install is incorrect
Issue #268: Cube flux from file error
Issue #85: Python does not start in Python 3
Issue #273: ha.value bug
Issue #277: Ticks for log normalized colorbar
Issue #275: logger crashes on warning when other loggers try to log
Issue #258: 422 Invalid Parameters
Issue #271: Problem in dowloading image.
Issue #97: sqlalchemy-boolean-search not found when installed from pip source
Issue #227: Marvin installation in python 3.6 (update setuptools to 36)
Issue #262: problem with marvin update
Issue #270: BPT array sizing not compatible
Issue #88: Deployment at Utah requires automatisation
Issue #234: Add (and use) functions to the datamodel to determine plotting parameters
Issue #278: marvin_test_if decorator breaks in python 2.7
Issue #274: cube slicing to get a spaxel fails with maps error
Issue #39: implement more complete testing framework
Issue #242: Result object representation error with 0 query results
Issue #159: Marvin issues multiple warnings in PY3
Issue #149: Improve integrated flux maps display in web
[2.1.3] - 2017/05/18¶
Issue #204: added elpetro_absmag colours to mangaSampleDB models.
Issue #253: Plotting tutorial.
Issue #223: Easy multi-panel map plotting (with correctly placed colorbars).
Issue #232 and Issue #251: Uses matplotlib style sheets context managers for plotting (map, spectrum, and BPT) and restores previous defaults before methods finish.
Issue #189: Map plotting accepts user-defined value, ivar, and/or mask (including BPT masks).
Issue #252: Quantile clipping for properties other than velocity, sigma, or flux in web.
Issue #243: inverted
__getitem__behaviour for Cube/Maps/ModelCube and fixed tests.
Modified Flask Profiler File to always point to $MARVIN_DIR/flask_profiler.sql
Issue #241: Moved map plotting methods from tools/map to utils/plot/map
Issue #229 and Issue #231: Switch to new gray/hatching scheme (in tools and web):
gray: spaxels with NOCOV.
hatched: spaxels with bad data (UNRELIABLE and DONOTUSE) or S/N below some minimum value.
colored: good data.
Issue #238: Move plot defaults to datamodel (i.e., bitmasks, colormaps, percentile clips, symmetric, minimum SNR).
Issue #206: SNR minimum to None (effectively 0) for velocity maps so that they aren’t hatched near the zero velocity contour.
Simplified default colormap name to “linearlab.”
Decreased map plot title font size in web so that it does not run onto second line and overlap plot.
Interactive prompt for username in sdss_access now works for Python 3.
Fixed #195: The data file for the default colormap for
Map.plot()(“linear_Lab”) is now included in pip version of Marvin and does not throw invalid
FileNotFoundErrorif the data file is missing.
Fixed #143: prevents access mode to go in to remote if filename is present.
Fixed #213: shortcuts are now only applied on full words, to avoid blind replacements.
Fixed #206: no longer masks spaxels close to zero velocity contour in web and tools map plots
Fixed #229: corrects web bitmask parsing for map plots
Fixed #231: hatch regions within IFU but without data in map plots
Fixed #255: Lean tutorial code cells did not work with the ipython directive, so they now use the python directive.
Highcharts draggable legend cdn.
[2.1.2] - 2017/03/17¶
API and Web argument validation using webargs and marshmallow. If parameters invalid, returns 422 status.
Per Issue #186: Switched to using the elpetro version of stellar mass, absolute magnitude i-band, and i-band mass-to-light ratio for NSA web display, from sersic values. (elpetro_logmass, elpetro_absmag_i, elpetro_mtol_i)
Issue #188: deprecated snr in favour of snr_min for get_bpt. snr can still be used.
Issue #187: Renamed NSA Display tab in web to Galaxy Properties. Added a link to the NASA-Sloan Atlas catalogue to the table title.
Moved our documentation to readthedocs for version control. Updated all Marvin web documenation links to point to readthedocs.
[2.1.1] - 2017/02/18¶
Added query runtime output in search page html. And a warning if query is larger than 20 seconds.
Removed the python 3 raise Exception in the check_marvin bin
Reverted the api/query return output from jsonify back to json.dumps
This is an issue with python 2.7.3 namedtuple vs 2.7.11+
[2.1.0] - 2017/02/16¶
Restructured documentation index page.
Improved installation documentation:
Removed old installation text
Added section on marvin SDSS dependencies and SAS_BASE_DIR
Added section for FAQ about installation
Added web browser cache issue into FAQ
Added traceback info in the API calls
Added traceback attribute in Brain config
Added hidden _traceback attribute in Marvin config
Only implemented in two Query API calls at the moment
Added a few tests for traceback
see usage in cube_query in marvin/api/query.py
Added the Ha_to_Hb ratio the DAP ModelClasses for querying
Added new script to perform somce basic system, os, and Marvin checks: bin/check_marvin
Added an alert banner when the user is using Safari. See #94.
Issue #122: added ra/dec to spaxel
Issue #145: Limited the number of query parameters in the web
Added more tests to Results for sorting, paging, and getting subsets
Added kwargs input for Spaxel when using Result.convertToTool
Added automatic Sentry error logging #147 into MarvinError, and Sentry in Flask for production mode
Added custom error handlers for the web, with potential user feedback form
Added Sentry tool for grabbing and displaying Sentry statistics
Added text to MarvinError with a Github Issues link and description of how to submit and issue
Added Results option to save to CSV
Added new parameters in Marvin Config to turn off Sentry error handling and Github Issue message
Added Python example code for getting a spectrum in galaxy page of web.
Added new test for image utilities getRandomImages, getImagesByPlate, getImagesByList
Added new documentation on Image Utilities
Added new image utility function showImage, which displays images from your local SAS
Added the Kewley+06 implementation of the BPT classification as
Added quick access to the NSA information for a Cube/Maps either from mangaSampleDB or drpall.
When marvin is running from source (not dist),
Removed the cleanUpQueries method to assess db stability
Switched dogpile.cache from using a file to python-memcached
Syntax changes and bug fixes to get Marvin Web working when Marvin run on 3.5
Got Queries and Results working in 3.5
Changed all convertToTool options in Results from mangaid to plateifu
Added release explicitly into api query routes
Modified the decision tree in query to throw an error in local mode
Modified convertToTool to accept a mode keyword
Modifed the MarvinError for optional Sentry exception catching, and github issue inclusion
Updated all Marvin tests to turn off Sentry exception catching and the github message
Updated some of the Tools Snippets on the web
Overhauled Map plotting
uses DAP bitmasks (NOVALUE, BADVALUE, MATHERROR, BADFIT, and DONOTUSE)
adds percentile and sigma clipping
adds hatching for regions with data (i.e., a spectrum) but no measurement by the DAP
adds Linear Lab color map
adds option for logarithmic colorbar
adds option to use sky coordinates
adds map property name as title
makes plot square
sets plotting defaults:
cmap is linear_Lab (sequential)
cmap is RdBu_r (diverging) for velocity plots (Note: this is reversed from the sense of the default coolwarm colormap in v2.0—red for positive velocities and blue for negative velocities)
cmap is inferno (sequential) for sigma plots
clips at 5th and 95th percentiles
clips at 10th and 90th percentiles for velocity and sigma plots
velocity plots are symmetric about 0
uses DAP bitmasks NOVALUE, BADVALUE, MATHERROR, BADFIT, and DONOTUSE
also masks spaxels with ivar=0
minimum SNR is 1
Changed Marvin Plate path back to the standard MarvinToolsClass use
Made sdss_access somewhat more Python 3 compatible
Modified the image utilities to return local paths in local/remote modes and url paths when as_url is True
downloadList utility function now downloads images
updated the limit-as parameter in the uwsgi ini file to 4096 mb from 1024 mb for production environment
Issue #115: drpall does not get updated when a tool sets a custom release.
Issue #107: missing os library under save function of Map class
Issue #117: hybrid colours were incorrect as they were being derived from petroth50_el.
Issue #119: test_get_spaxel_no_db fails
Issue #121: bugfix with misspelled word in downloadList utility function
Issue #105: query results convertToTool not robust when null/default parameters not present
Issue #136: BinTest errors when nose2 run in py3.5 and marvin server in 3.5
Issue #137: PIL should work in py2.7 and py3.5
Issue #172: broken mode=auto in image utilities
Issue #158: version discrepancy in setup.py
[2.0.9] - 2016/11/19¶
Docs now use
[2.0.6] - 2016/11/17¶
Bug in Queries with dap query check running in remote mode. Param form is empty.
[2.0.5] - 2016/11/17¶
Added netrc configuration to installation documentation.
Added netrc check on init.
Added mask to model spaxel.
Bug in Cube tool when a galaxy loaded from db does not have NSA info; no failure with redshift
Two bugs in index.py on KeyErrors: Sentry issues 181369719,181012809
Bug on plate web page preventing meta-data from rendering
Fixed installation in Python 3.
Fixed long_description in setup.py to work with PyPI.
Fixed a problem that made marvin always use the modules in extern