General reading and writing base classes (sarpy.io.general.base)

The basic definitions for file-like reading and writing. This is generally centered on image-like file efforts, and array-like interaction with image data.

This module completely revamped in version 1.3.0 for data segment usage.

sarpy.io.general.base.READER_TYPES = ('SICD', 'SIDD', 'CPHD', 'CRSD', 'OTHER')

The reader_type enum

exception sarpy.io.general.base.SarpyIOError

Bases: SarpyError

A custom exception class for discovered input/output errors.

add_note()

Exception.add_note(note) – add a note to the exception

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

sarpy.io.general.base.check_for_openers(start_package: str, register_method: Callable) → None

Walks the package, and registers the discovered openers. That is, the modules with an is_a() method.

Parameters:

• start_package (str) –

• register_method (Callable) –

class sarpy.io.general.base.BaseReader(data_segment: None | DataSegment | Sequence[DataSegment], reader_type: str = 'OTHER', close_segments: bool = True, delete_files: str | Sequence[str] | None = None)

Bases: object

The basic reader definition, using array-like data fetching.

Changed in version 1.3.0

property file_name: str | None

Defined as a convenience property.

Type:

None|str

property reader_type: str

A descriptive string for the type of reader

Type:

str

property data_segment: DataSegment | Tuple[DataSegment, ...]

The data segment collection.

Type:

DataSegment|Tuple[DataSegment, …]

property image_count: int

The number of images/data segments from which to read.

Type:

int

get_data_segment_as_tuple() → Tuple[DataSegment, ...]

Get the data segment collection as a tuple, to avoid the need for redundant checking issues.

Return type:

Tuple[DataSegment, …]

property data_size: Tuple[int, ...] | Tuple[Tuple[int, ...]]

the output/formatted data size(s) of the data segment(s). If there is a single data segment, then this will be Tuple[int, …], otherwise it will be Tuple[Tuple, int, …], …].

Type:

Tuple[int, …]|Tuple[Tuple[int, …], …]

get_data_size_as_tuple() → Tuple[Tuple[int, ...], ...]

Get the data size collection as a tuple of tuples, to avoid the need for redundant checking issues.

Return type:

Tuple[Tuple[int, …], …]

property raw_data_size: Tuple[int, ...] | Tuple[Tuple[int, ...]]

the raw data size(s) of the data segment(s). If there is a single data segment, then this will be Tuple[int, …], otherwise it will be Tuple[Tuple, int, …], …].

Type:

Tuple[int, …]|Tuple[Tuple[int, …], …]

get_raw_data_size_as_tuple() → Tuple[Tuple[int, ...], ...]

Get the raw data size collection as a tuple of tuples, to avoid the need for redundant checking issues.

Return type:

Tuple[Tuple[int, …], …]

property files_to_delete_on_close: List[str]

A collection of files to delete on the close operation.

Type:

List[str]

property closed: bool

Is the reader closed? Reading will result in a ValueError

Type:

bool

read_chip(*ranges: Sequence[None | int | Tuple[int, ...] | slice], index: int = 0, squeeze: bool = True) → ndarray

This is identical to read(), and presented for backwards compatibility.

Parameters:

• ranges (Sequence[Union[None, int, Tuple[int, ...], slice]]) –

• index (int) –

• squeeze (bool) –

Return type:

numpy.ndarray

See also

read()

read(*ranges: None | int | Tuple[int, ...] | slice, index: int = 0, squeeze: bool = True) → ndarray

Read formatted data from the given data segment. Note this is an alias to the __call__() called as reader(*ranges, index=index, raw=False, squeeze=squeeze).

Parameters:

• ranges (Sequence[Union[None, int, Tuple[int, ...], slice]]) – The slice definition appropriate for data_segment[index].read() usage.

• index (int) – The data_segment index. This is ignored if image_count== 1.

• squeeze (bool) – Squeeze length 1 dimensions out of the shape of the return array?

Return type:

numpy.ndarray

See also

See

meth:sarpy.io.general.data_segment.DataSegment.read.

read_raw(*ranges: None | int | Tuple[int, ...] | slice, index: int = 0, squeeze: bool = True) → ndarray

Read raw data from the given data segment. Note this is an alias to the __call__() called as reader(*ranges, index=index, raw=True, squeeze=squeeze).

Parameters:

• ranges (Sequence[Union[None, int, Tuple[int, ...], slice]]) – The slice definition appropriate for data_segment[index].read() usage.

• index (int) – The data_segment index. This is ignored if image_count== 1.

• squeeze (bool) – Squeeze length 1 dimensions out of the shape of the return array?

Return type:

numpy.ndarray

See also

See

meth:sarpy.io.general.data_segment.DataSegment.read_raw.

close() → None

This should perform any necessary clean-up operations, like closing open file handles, deleting any temp files, etc.

class sarpy.io.general.base.FlatReader(underlying_array: ndarray, reader_type: str = 'OTHER', formatted_dtype: str | dtype | None = None, formatted_shape: Tuple[int, ...] | None = None, reverse_axes: int | Sequence[int] | None = None, transpose_axes: Tuple[int, ...] | None = None, format_function: FormatFunction | None = None, close_segments: bool = True)

Bases: BaseReader

Class for passing a numpy array straight through as a reader.

Changed in version 1.3.0

close() → None

This should perform any necessary clean-up operations, like closing open file handles, deleting any temp files, etc.

property closed: bool

Is the reader closed? Reading will result in a ValueError

Type:

bool

property data_segment: DataSegment | Tuple[DataSegment, ...]

The data segment collection.

Type:

DataSegment|Tuple[DataSegment, …]

property data_size: Tuple[int, ...] | Tuple[Tuple[int, ...]]

the output/formatted data size(s) of the data segment(s). If there is a single data segment, then this will be Tuple[int, …], otherwise it will be Tuple[Tuple, int, …], …].

Type:

Tuple[int, …]|Tuple[Tuple[int, …], …]

property file_name: str | None

Defined as a convenience property.

Type:

None|str

property files_to_delete_on_close: List[str]

A collection of files to delete on the close operation.

Type:

List[str]

get_data_segment_as_tuple() → Tuple[DataSegment, ...]

Get the data segment collection as a tuple, to avoid the need for redundant checking issues.

Return type:

Tuple[DataSegment, …]

get_data_size_as_tuple() → Tuple[Tuple[int, ...], ...]

Get the data size collection as a tuple of tuples, to avoid the need for redundant checking issues.

Return type:

Tuple[Tuple[int, …], …]

get_raw_data_size_as_tuple() → Tuple[Tuple[int, ...], ...]

Get the raw data size collection as a tuple of tuples, to avoid the need for redundant checking issues.

Return type:

Tuple[Tuple[int, …], …]

property image_count: int

The number of images/data segments from which to read.

Type:

int

property raw_data_size: Tuple[int, ...] | Tuple[Tuple[int, ...]]

the raw data size(s) of the data segment(s). If there is a single data segment, then this will be Tuple[int, …], otherwise it will be Tuple[Tuple, int, …], …].

Type:

Tuple[int, …]|Tuple[Tuple[int, …], …]

read(*ranges: None | int | Tuple[int, ...] | slice, index: int = 0, squeeze: bool = True) → ndarray

Read formatted data from the given data segment. Note this is an alias to the __call__() called as reader(*ranges, index=index, raw=False, squeeze=squeeze).

Parameters:

• ranges (Sequence[Union[None, int, Tuple[int, ...], slice]]) – The slice definition appropriate for data_segment[index].read() usage.

• index (int) – The data_segment index. This is ignored if image_count== 1.

• squeeze (bool) – Squeeze length 1 dimensions out of the shape of the return array?

Return type:

numpy.ndarray

See also

See

meth:sarpy.io.general.data_segment.DataSegment.read.

read_chip(*ranges: Sequence[None | int | Tuple[int, ...] | slice], index: int = 0, squeeze: bool = True) → ndarray

This is identical to read(), and presented for backwards compatibility.

Parameters:

• ranges (Sequence[Union[None, int, Tuple[int, ...], slice]]) –

• index (int) –

• squeeze (bool) –

Return type:

numpy.ndarray

See also

read()

read_raw(*ranges: None | int | Tuple[int, ...] | slice, index: int = 0, squeeze: bool = True) → ndarray

Read raw data from the given data segment. Note this is an alias to the __call__() called as reader(*ranges, index=index, raw=True, squeeze=squeeze).

Parameters:

• ranges (Sequence[Union[None, int, Tuple[int, ...], slice]]) – The slice definition appropriate for data_segment[index].read() usage.

• index (int) – The data_segment index. This is ignored if image_count== 1.

• squeeze (bool) – Squeeze length 1 dimensions out of the shape of the return array?

Return type:

numpy.ndarray

See also

See

meth:sarpy.io.general.data_segment.DataSegment.read_raw.

property reader_type: str

A descriptive string for the type of reader

Type:

str

class sarpy.io.general.base.AggregateReader(readers: Sequence[BaseReader], reader_type: str = 'OTHER', close_readers: bool = False)

Bases: BaseReader

Aggregate multiple readers into a single reader instance. This default aggregate implementation will not preserve any other metadata structures.

property index_mapping: Tuple[Tuple[int, int]]

The index mapping of the form (reader index, segment index in reader).

Type:

Tuple[Tuple[int, int]]

close() → None

This should perform any necessary clean-up operations, like closing open file handles, deleting any temp files, etc.

property closed: bool

Is the reader closed? Reading will result in a ValueError

Type:

bool

property data_segment: DataSegment | Tuple[DataSegment, ...]

The data segment collection.

Type:

DataSegment|Tuple[DataSegment, …]

property data_size: Tuple[int, ...] | Tuple[Tuple[int, ...]]

the output/formatted data size(s) of the data segment(s). If there is a single data segment, then this will be Tuple[int, …], otherwise it will be Tuple[Tuple, int, …], …].

Type:

Tuple[int, …]|Tuple[Tuple[int, …], …]

property file_name: str | None

Defined as a convenience property.

Type:

None|str

property files_to_delete_on_close: List[str]

A collection of files to delete on the close operation.

Type:

List[str]

get_data_segment_as_tuple() → Tuple[DataSegment, ...]

Get the data segment collection as a tuple, to avoid the need for redundant checking issues.

Return type:

Tuple[DataSegment, …]

get_data_size_as_tuple() → Tuple[Tuple[int, ...], ...]

Get the data size collection as a tuple of tuples, to avoid the need for redundant checking issues.

Return type:

Tuple[Tuple[int, …], …]

get_raw_data_size_as_tuple() → Tuple[Tuple[int, ...], ...]

Get the raw data size collection as a tuple of tuples, to avoid the need for redundant checking issues.

Return type:

Tuple[Tuple[int, …], …]

property image_count: int

The number of images/data segments from which to read.

Type:

int

property raw_data_size: Tuple[int, ...] | Tuple[Tuple[int, ...]]

the raw data size(s) of the data segment(s). If there is a single data segment, then this will be Tuple[int, …], otherwise it will be Tuple[Tuple, int, …], …].

Type:

Tuple[int, …]|Tuple[Tuple[int, …], …]

read(*ranges: None | int | Tuple[int, ...] | slice, index: int = 0, squeeze: bool = True) → ndarray

Read formatted data from the given data segment. Note this is an alias to the __call__() called as reader(*ranges, index=index, raw=False, squeeze=squeeze).

Parameters:

• ranges (Sequence[Union[None, int, Tuple[int, ...], slice]]) – The slice definition appropriate for data_segment[index].read() usage.

• index (int) – The data_segment index. This is ignored if image_count== 1.

• squeeze (bool) – Squeeze length 1 dimensions out of the shape of the return array?

Return type:

numpy.ndarray

See also

See

meth:sarpy.io.general.data_segment.DataSegment.read.

read_chip(*ranges: Sequence[None | int | Tuple[int, ...] | slice], index: int = 0, squeeze: bool = True) → ndarray

This is identical to read(), and presented for backwards compatibility.

Parameters:

• ranges (Sequence[Union[None, int, Tuple[int, ...], slice]]) –

• index (int) –

• squeeze (bool) –

Return type:

numpy.ndarray

See also

read()

read_raw(*ranges: None | int | Tuple[int, ...] | slice, index: int = 0, squeeze: bool = True) → ndarray

Read raw data from the given data segment. Note this is an alias to the __call__() called as reader(*ranges, index=index, raw=True, squeeze=squeeze).

Parameters:

• ranges (Sequence[Union[None, int, Tuple[int, ...], slice]]) – The slice definition appropriate for data_segment[index].read() usage.

• index (int) – The data_segment index. This is ignored if image_count== 1.

• squeeze (bool) – Squeeze length 1 dimensions out of the shape of the return array?

Return type:

numpy.ndarray

See also

See

meth:sarpy.io.general.data_segment.DataSegment.read_raw.

property reader_type: str

A descriptive string for the type of reader

Type:

str

class sarpy.io.general.base.BaseWriter(data_segment: DataSegment | Sequence[DataSegment])

Bases: object

Writer definition, using array-like data writing.

Introduced in version 1.3.0

property file_name: str | None

Defined as a convenience property.

Type:

None|str

property data_segment: Tuple[DataSegment, ...]

The data segment collection.

Type:

Tuple[DataSegment, …]

property image_count: int

The number of overall images/data segments.

Type:

int

property data_size: Tuple[Tuple[int, ...]]

the formatted data sizes of the data segments.

Type:

Tuple[Tuple[int, …], …]

property raw_data_size: Tuple[int, ...] | Tuple[Tuple[int, ...]]

the raw data sizes of the data segments.

Type:

Tuple[Tuple[int, …], …]

property closed: bool

Is the writer closed? Reading file after writing can result in a ValueError if writer was not closed.

Type:

bool

write_chip(data: ndarray, start_indices: int | Tuple[int, ...] | None = None, subscript: Tuple[slice, ...] | None = None, index: int = 0) → None

This is identical to write(), and presented for backwards compatibility.

Parameters:

• data (numpy.ndarray) –

• start_indices (None|int|Tuple[int, ...]) –

• subscript (None|Tuple[slice, ...]) –

• index (int) –

See also

See

meth:sarpy.io.general.data_segment.DataSegment.write.

write(data: ndarray, start_indices: int | Tuple[int, ...] | None = None, subscript: Tuple[slice, ...] | None = None, index: int = 0) → None

Write the data to the appropriate data segment. This is an alias to writer(data, start_indices=start_indices, subscript=subscript, index=index, raw=False).

Only one of `start_indices` and `subscript` should be specified.

Parameters:

• data (numpy.ndarray) – The data to write.

• start_indices (None|int|Tuple[int, ...]) – Assuming a contiguous chunk of data, this provides the starting indices of the chunk. Any missing (tail) coordinates will be filled in with 0’s.

• subscript (None|Tuple[slice, ...]) – In contrast to providing start_indices, the slicing definition in formatted coordinates pertinent to the specified data segment.

• index (int) – The index of the

See also

See

meth:sarpy.io.general.data_segment.DataSegment.write.

write_raw(data: ndarray, start_indices: int | Tuple[int, ...] | None = None, subscript: Tuple[slice, ...] | None = None, index: int = 0) → None

Write the raw data to the file(s). This is an alias to writer(data, start_indices=start_indices, subscript=subscript, index=index, raw=True).

Only one of `start_indices` and `subscript` should be specified.

Parameters:

• data (numpy.ndarray) – The data to write.

• start_indices (None|int|Tuple[int, ...]) – Assuming a contiguous chunk of data, this provides the starting indices of the chunk. Any missing (tail) coordinates will be filled in with 0’s.

• subscript (None|Tuple[slice, ...]) – In contrast to providing start_indices, the slicing definition in raw coordinates pertinent to the specified data segment.

• index (int) –

See also

See

meth:sarpy.io.general.data_segment.DataSegment.write_raw.

flush(force: bool = False) → None

Try to perform any necessary steps to flush written data to the disk/buffer.

Parameters:

force (bool) – Try force flushing, even for incompletely written data.

Return type:

None

close() → None

This should perform any necessary final steps, like closing open file handles, deleting any temp files, etc. Trying to read newly created file without closing may raise a ValueError.