Module Documentation

fitsmap.convert module

Converts image files and catalogs into a leafletJS map.

fitsmap.convert.balance_array(array: numpy.ndarray) → fitsmap.padded_array.PaddedArray

Pads input array with zeros so that the long side is a multiple of the short side.

Parameters:array (np.ndarray) – array to balance Returns:a balanced version of array via a PaddedArray

fitsmap.convert.build_mpl_objects(array: numpy.ndarray, norm_kwargs: Dict[str, Any]) → Tuple[matplotlib.colors.Normalize, matplotlib.colors.Colormap]

Builds the matplotlib objects that norm and convert fits data to RGB.

Parameters:

• array (np.ndarray) – The array to be tiled
• norm_kwargs (Dict[str, Any]) – The kwargs to be passed to simple_norm

Returns:

The matplotlib objects

needed to create a tile

Return type:

Tuple[mpl.colors.Normalize, mpl.colors.Colormap]

fitsmap.convert.build_path(z, y, x, out_dir) → str

Maps zoom and coordinate location to a subdir in out_dir

Parameters:

• z (int) – The zoom level for the tiles
• y (int) – The zoom level for the tiles
• x (int) – The zoom level for the tiles
• out_dir (str) – The root directory the tiles are saved in

Returns:

The str path to save the tile in

fitsmap.convert.dir_to_map(directory: str, out_dir: str = '.', exclude_predicate: Callable = <function <lambda>>, title: str = 'FitsMap', task_procs: int = 0, procs_per_task: int = 0, catalog_delim: str = ',', cat_wcs_fits_file: str = None, max_catalog_zoom: int = -1, tile_size: Tuple[int, int] = [256, 256], norm_kwargs: Union[Dict[str, Any], Dict[str, Dict[str, Any]]] = {}, rows_per_column: int = inf, prefer_xy: bool = False, catalog_starts_at_one: bool = True, img_tile_batch_size: int = 1000) → None

Converts a list of files into a LeafletJS map.

Parameters:

• directory (str) – Path to directory containing the files to be converted
• out_dir (str) – Directory to place the genreated web page and associated subdirectories
• exclude_predicate (Callable) – A function that is applied to every file in directory and returns True if the file should not be processed as a part of the map, and False if it should be processed
• title (str) – The title to placed on the webpage
• task_procs (int) – The number of tasks to run in parallel
• procs_per_task (int) – The number of tiles to process in parallel
• catalog_delim (str) – The delimiter for catalog (.cat) files. Deault is comma.
• cat_wcs_fits_file (str) – A fits file that has the WCS that will be used to map ra and dec coordinates from the catalog files to x and y coordinates in the map. Note, that this file isn’t subject to the exlclude_predicate, so you can exclude a fits file from being tiled, but still use its header for WCS.
• max_catalog_zoom (int) – The zoom level to stop clustering on, the default is the max zoom level of the image. For images with a high source density, setting this higher than the max zoom will help with performance.
• tile_size (Tuple[int, int]) – The tile size for the leaflet map. Currently only [256, 256] is supported.
• norm_kwargs (Union[Dict[str, Any], Dict[str, Dict[str, Any]]]) – Optional normalization keyword arguments passed to astropy.visualization.simple_norm. Can either be a single dictionary of keyword arguments, or a dictionary of keyword arguments for each image where the keys are the image names not full paths. The default is linear scaling using min/max values. See documentation for more information: https://docs.astropy.org/en/stable/api/astropy.visualization.mpl_normalize.simple_norm.html
• rows_per_column (int) – If converting a catalog, the number of items in have in each column of the marker popup. By default produces all values in a single column. Setting this value can make it easier to work with catalogs that have a lot of values for each object.
• prefer_xy (bool) – If True x/y coordinates should be preferred if both ra/dec and x/y are present in a catalog
• catalog_starts_at_one (bool) – True if the catalog is 1 indexed, False if the catalog is 0 indexed
• img_tile_batch_size (int) – The number of image tiles to process in parallel when task_procs > 1

Example of image specific norm_kwargs vs single norm_kwargs:

>>> norm_kwargs = {
>>>    "test.fits": {"stretch": "log", "min_percent": 1, "max_percent": 99.5},
>>>    "test2.fits": {"stretch": "linear", "min_percent": 5, "max_percent": 99.5},
>>> }
>>> # or
>>> norm_kwargs = {"stretch": "log", "min_percent": 1, "max_percent": 99.5}

Returns:

None

Raises:

• ValueError if the dir is empty, there are no convertable files or if
• exclude_predicate exlcudes all files

fitsmap.convert.files_to_map(files: List[str], out_dir: str = '.', title: str = 'FitsMap', task_procs: int = 0, procs_per_task: int = 0, catalog_delim: str = ',', cat_wcs_fits_file: str = None, max_catalog_zoom: int = -1, tile_size: Tuple[int, int] = [256, 256], norm_kwargs: dict = {}, rows_per_column: int = inf, prefer_xy: bool = False, catalog_starts_at_one: bool = True, img_tile_batch_size: int = 1000) → None

Converts a list of files into a LeafletJS map.

Parameters:

• files (List[str]) – List of files to convert into a map, can include image files (.fits, .png, .jpg) and catalog files (.cat)
• out_dir (str) – Directory to place the genreated web page and associated subdirectories
• title (str) – The title to placed on the webpage
• task_procs (int) – The number of tasks to run in parallel
• procs_per_task (int) – The number of tiles to process in parallel
• catalog_delim (str) – The delimited for catalog (.cat) files. Deault is whitespace.
• cat_wcs_fits_file (str) – A fits file that has the WCS that will be used to map ra and dec coordinates from the catalog files to x and y coordinates in the map
• max_catalog_zoom (int) – The zoom level to stop clustering on, the default is the max zoom level of the image. For images with a high source density, setting this higher than the max zoom will help with performance.
• tile_size (Tuple[int, int]) – The tile size for the leaflet map. Currently only [256, 256] is supported.
• norm_kwargs (Union[Dict[str, Any], Dict[str, Dict[str, Any]]]) – Optional normalization keyword arguments passed to astropy.visualization.simple_norm. Can either be a single dictionary of keyword arguments, or a dictionary of keyword arguments for each image where the keys are the image names not full paths. The default is linear scaling using min/max values. See documentation for more information: https://docs.astropy.org/en/stable/api/astropy.visualization.mpl_normalize.simple_norm.html
• rows_per_column (int) – If converting a catalog, the number of items in have in each column of the marker popup. By default produces all values in a single column. Setting this value can make it easier to work with catalogs that have a lot of values for each object.
• prefer_xy (bool) – If True x/y coordinates should be preferred if both ra/dec and x/y are present in a catalog
• catalog_starts_at_one (bool) – True if the catalog is 1 indexed, False if the catalog is 0 indexed
• img_tile_batch_size (int) – The number of image tiles to process in parallel when task_procs > 1

Example of image specific norm_kwargs vs single norm_kwargs:

>>> norm_kwargs = {
>>>    "test.fits": {"stretch": "log", "min_percent": 1, "max_percent": 99.5},
>>>    "test2.fits": {"stretch": "linear", "min_percent": 5, "max_percent": 99.5},
>>> }
>>> # or
>>> norm_kwargs = {"stretch": "log", "min_percent": 1, "max_percent": 99.5}

Returns:None

fitsmap.convert.filter_on_extension(files: List[str], extensions: List[str], exclude_predicate: Callable = None) → List[str]

Filters out files from files based on extensions and exclude_predicate

Parameters:

• files (List[str]) – A list of file paths to be filtered
• extensions (List[str]) – List of extensions to filter files on
• exclude_predicate (Callable) – A function that accepts a single str as input and returns a True if the file should be excluded, and False if it should be included

Returns:

A list of files which have an extension thats in extensions and for which exclude_predicate(file)==False`

fitsmap.convert.get_array(file_location: str) → fitsmap.padded_array.PaddedArray

Opens the array at file_location can be an image or a fits file

Parameters:file_location (str) – the path to the image Returns:A numpy array representing the image.

fitsmap.convert.get_map_layer_name(file_location: str) → str

Tranforms a file_location into the javascript layer name.

Parameters:file_location (str) – The file location to convert Returns:The javascript name that will be used in the HTML map

fitsmap.convert.get_marker_file_name(file_location: str) → str

Tranforms a file_location into the javascript marker file name.

Parameters:file_location (str) – The file location to convert Returns:The javascript name that will be used in the HTML map

fitsmap.convert.get_total_tiles(min_zoom: int, max_zoom: int) → int

Returns the total number of tiles that will be generated from an image.

Parameters:

• min_zoom (int) – The minimum zoom level te image will be tiled at
• max_zoom (int) – The maximum zoom level te image will be tiled at

Returns:

The total number of tiles that will be generated

fitsmap.convert.get_zoom_range(shape: Tuple[int, int], tile_size: Tuple[int, int]) → Tuple[int, int]

Returns the supported native zoom range for an give image size and tile size.

Parameters:

• shape (Tuple[int, int]) – The shape that is going to be tiled
• tile_size (Tuple[int, int]) – The size of the image tiles

Returns:

A tuple containing the (minimum zoom level, maximum zoom level)

fitsmap.convert.imread_default(path: str, default: numpy.ndarray) → numpy.ndarray

Opens an image if it exists, if not returns a tranparent image. :param path: Image file location :type path: str :param size: The image size, assumed square. Defaults to 256. :type size: int, optional

Returns:

the image if it exists. if not, a transparent image of

size (size, size, 4).

Return type:np.ndarray

fitsmap.convert.line_to_cols(raw_col_vals: str) → List[str]

Transform a raw text line of column names into a list of column names

Parameters:raw_line (str) – String from textfile Returns:A list of the column names in order

fitsmap.convert.line_to_json(wcs: astropy.wcs.wcs.WCS, columns: List[str], catalog_assets_path: str, src_vals: List[str], catalog_starts_at_one: bool = 1) → Dict[str, Any]

Transform a raw text line attribute values into a JSON marker

Parameters:raw_line (str) – String from the marker file Returns:A list of the column names in order

fitsmap.convert.make_dirs(out_dir: str, min_zoom: int, max_zoom: int) → None

Builds the directory tree for storing image tiles.

Parameters:

• out_dir (str) – The root directory to generate the tree in
• min_zoom (int) – The minimum zoom level the image will be tiled at
• max_zoom (int) – The maximum zoom level the image will be tiled at

Returns:

None

fitsmap.convert.make_marker_tile(cluster: fitsmap.supercluster.Supercluster, out_dir: str, job: Union[Tuple[int, Tuple[int, int]], List[Tuple[int, Tuple[int, int]]]]) → None

fitsmap.convert.make_tile_mpl(mpl_norm: matplotlib.colors.Normalize, mpl_cmap: matplotlib.colors.Colormap, tile: numpy.ndarray) → numpy.ndarray

Converts array data into an image using matplotlib :param mpl_f: The matplotlib figure to use :type mpl_f: mpl.figure.Figure :param mpl_img: The matplotlib image to use :type mpl_img: mpl.image.AxesImage :param mpl_alpha_f: A function that

converts the input array into an RGBA

Parameters:tile (np.ndarray) – The array data Returns:The array data converted into an image using Matplotlib Return type:np.ndarray

fitsmap.convert.make_tile_pil(tile: numpy.ndarray) → numpy.ndarray

Converts the input array into an image using PIL :param tile: The array data to be converted :type tile: np.ndarray

Returns:an RGBA version of the input data Return type:np.ndarray

fitsmap.convert.mem_safe_make_tile(out_dir: str, tile_f: Callable[[numpy.ndarray], numpy.ndarray], array: numpy.ndarray, job: Union[Tuple[int, int, int, slice, slice], List[Tuple[int, int, int, slice, slice]]]) → None

Extracts a tile from array and saves it at the proper place in out_dir using PIL. :param out_dir: The directory to save tile in :type out_dir: str :param tile_f: A function that converts a

subset of the image array into an png tile. Is one of make_tile_pil or make_tile_mpl

Parameters:

• array (np.ndarray) – Array to extract a slice from
• job (Tuple[int, int, int, slice, slice]) – A tuple containing z, y, x, dim0_slices, dim1_slices. Where (z, y, x) define the zoom and the coordinates, and (dim0_slices, and dim1_slices) are slice objects that extract the tile.

Returns:

None

fitsmap.convert.process_catalog_file_chunk(process_f: Callable, fname: str, delimiter: str, q: ray.util.queue.Queue, start: int, end: int) → List[dict]

fitsmap.convert.slice_idx_generator(shape: Tuple[int, int], zoom: int, tile_size: int) → Iterable[Tuple[int, int, int, slice, slice]]

fitsmap.convert.tile_img(file_location: str, pbar_ref: Tuple[int, ray.util.queue.Queue], tile_size: Tuple[int, int] = [256, 256], min_zoom: int = 0, out_dir: str = '.', mp_procs: int = 0, norm_kwargs: dict = {}, batch_size: int = 1000) → None

Extracts tiles from the array at file_location.

Parameters:

• file_location (str) – The file location of the image to tile
• pbar_loc (int) – The index of the location of to print the tqdm bar
• tile_size (Tuple[int, int]) – The pixel size of the tiles in the map
• min_zoom (int) – The minimum zoom to create tiles for. The default value is 0, but if it can be helpful to set it to a value greater than zero if your running out of memory as the lowest zoom images can be the most memory intensive.
• out_dir (str) – The root directory to save the tiles in
• mp_procs (int) – The number of multiprocessing processes to use for generating tiles.
• norm_kwargs (dict) – Optional normalization keyword arguments passed to astropy.visualization.simple_norm. The default is linear scaling using min/max values. See documentation for more information: https://docs.astropy.org/en/stable/api/astropy.visualization.mpl_normalize.simple_norm.html
• batch_size (int) – The number of tiles to process at a time, when tiling in parallel. The default is 1000.

Returns:

None

fitsmap.convert.tile_markers(wcs_file: str, out_dir: str, catalog_delim: str, mp_procs: int, prefer_xy: bool, min_zoom: int, max_zoom: int, tile_size: int, max_x: int, max_y: int, catalog_starts_at_one: bool, catalog_file: str, pbar_ref: Tuple[int, ray.util.queue.Queue], batch_size: int = 500) → None

fitsmap.cartographer module

Helper functions for creating a leaflet JS HTML map.

fitsmap.cartographer.build_conditional_css(out_dir: str) → str

fitsmap.cartographer.build_conditional_js(out_dir: str, include_markerjs: bool) → str

fitsmap.cartographer.build_html(title: str, extra_js: str, extra_css: str) → str

fitsmap.cartographer.build_index_js(image_layer_dicts: List[Dict[KT, VT]], marker_layer_dicts: List[str], rows_per_column: int, max_xy: Tuple[int, int]) → str

fitsmap.cartographer.build_urlCoords_js(img_wcs: astropy.wcs.wcs.WCS) → str

fitsmap.cartographer.cat_layer_dict_to_str(layer: dict, rows_per_column: int) → str

Convert layer dict to layer str for including in HTML file.

fitsmap.cartographer.chart(out_dir: str, title: str, img_layer_names: List[str], marker_layer_names: List[str], wcs: astropy.wcs.wcs.WCS, rows_per_column: int, max_xy: Tuple[int, int]) → None

Creates an HTML file containing a leaflet js map using the given params.

• Designed for internal use. Any method/variable can be deprecated/changed *
• without consideration. *

fitsmap.cartographer.extract_cd_matrix_as_string(wcs: astropy.wcs.wcs.WCS) → str

fitsmap.cartographer.get_colors() → Iterable[str]

fitsmap.cartographer.img_layer_dict_to_str(layer: dict) → str

Convert layer dict to layer str for including in HTML file.

fitsmap.cartographer.layer_name_to_dict(out_dir: str, max_zoom: int, min_zoom: int, max_native_zoom: int, name: str, color: str) → dict

Convert layer name to dict for conversion.

fitsmap.cartographer.leaflet_crs_js(tile_layers: List[dict]) → str

fitsmap.cartographer.leaflet_layer_control_declaration(img_layer_dicts: List[Dict[KT, VT]], cat_layer_dicts: List[Dict[KT, VT]]) → str

fitsmap.cartographer.leaflet_map_js(tile_layers: List[dict])

fitsmap.cartographer.leaflet_search_control_declaration(cat_layer_dicts: List[Dict[KT, VT]]) → str

fitsmap.cartographer.loading_screen_js(tile_layers: List[dict])

fitsmap.cartographer.move_support_images(out_dir: str) → List[str]