Data segment definition (sarpy.io.general.data_segment)¶

The object definitions for reading and writing data in single conceptual units using an interface based on slicing definitions and numpy arrays with formatting operations.

This module introduced in version 1.3.0.

Extracts any string elements (stripped and made all lowercase) from subscript entries.

Parameters:

subscript (None|str|int|slice|Sequence) –

Returns:

subscript (None|int|slice|Sequence) – With string entries removed
strings (Tuple[str, …]) – The string entries, stripped and made all lower case.

class sarpy.io.general.data_segment.DataSegment(raw_dtype: str | dtype, raw_shape: Tuple[int, ...], formatted_dtype: str | dtype, formatted_shape: Tuple[int, ...], reverse_axes: None | int | Sequence[int] = None, transpose_axes: Tuple[int, ...] | None = None, format_function: FormatFunction | None = None, mode: str = 'r')¶

Bases: object

Partially abstract base class representing one conceptual fragment of data read or written as an array. This is generally designed for images, but is general enough to support other usage.

Introduced in version 1.3.0.

Warning

The format function instance will be modified in place. Do not use the same format function instance across multiple data segments.

property raw_shape: Tuple[int, ...]¶

The raw shape.

Type:: Tuple[int, …]

property raw_ndim: int¶

The number of raw dimensions.

Type:: int

property mode: str¶

The mode.

Type:: str

property raw_dtype: dtype¶

The data type of the data returned by the read_raw() function.

Type:: numpy.dtype

property formatted_shape: Tuple[int, ...]¶

The formatted data shape.

Type:: Tuple[int, …]

property formatted_dtype: dtype¶

The data type of the formatted data, which will be returned by the read() function.

Type:: numpy.dtype

property formatted_ndim: int¶

The number of formatted dimensions.

Type:: int

property reverse_axes: Tuple[int, ...] | None¶

The collection of axes (with respect to raw order) along which we will reverse as part of transformation to formatted data order. If not None, then this will be a tuple in strictly increasing order.

Type:: None|Tuple[int, …]

property transpose_axes: Tuple[int, ...]¶

The transpose order for switching from raw order to formatted order, prior to applying any format function.

If populated, this must be a permutation of (0, 1, …, raw_ndim-1).

Type:: None|Tuple[int, …]

property format_function: FormatFunction¶

The format function which will be applied to the raw data.

Return type:: FormatFunction

property can_write_regular: bool¶

Can this data segment write regular data, which requires a function inverse?

Type:: bool

property closed: bool¶

Is the data segment closed? Reading or writing will result in a ValueError

Type:: bool

Verifies that the structure of the subscript is in keeping with the raw shape, and fills in any missing dimensions.

Parameters:: subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
Returns:: Guaranteed to be a tuple of slices of length raw_ndim.
Return type:: Tuple[slice, …]

Verifies that the structure of the subscript is in keeping with the formatted shape, and fills in any missing dimensions.

Parameters:: subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
Returns:: Guaranteed to be a tuple of slices of length formatted_ndim.
Return type:: Tuple[slice, …]

In keeping with data segment mode, read the data slice specified relative to the formatted data coordinates. This requires that mode is ‘r’.

Parameters:

subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
squeeze (bool) – Apply the numpy.squeeze operation, which eliminates dimension of size 1?

Return type:

numpy.ndarray

In keeping with data segment mode, read raw data from the source, without reformatting and or applying symmetry operations. This requires that mode is ‘r’.

Parameters:

subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) – These arguments are relative to raw data shape and order, no symmetry operations have been applied.
squeeze (bool) – Apply numpy.squeeze, which eliminates any dimensions of size 1?

Returns:

This will be of data type given by raw_dtype.

Return type:

numpy.ndarray

write(data: ndarray, start_indices: None | int | Tuple[int, ...] = None, subscript: None | Sequence[slice] = None, **kwargs) → None¶

In keeping with data segment mode, write the data provided in formatted form, assuming the slice specified relative to the formatted data coordinates.

This requires that mode is ‘w’, and format_function.has_inverse == True, because we have to apply the format function inverse to the provided data.

Only one of `start_indices` and `subscript` should be specified.

Parameters:

data (numpy.ndarray) – The data in formatted form, to be transferred to raw form and written.
start_indices (None|int|Tuple[int, ...]) – Assuming a contiguous chunk of data, this provides the starting indices of the chunk. Any missing (tail) coordinates will be filled in with 0’s.
subscript (None|Sequence[slice]) – The subscript definition in formatted coordinates.
kwargs –

Return type:

None

write_raw(data: ndarray, start_indices: None | int | Tuple[int, ...] = None, subscript: None | Sequence[slice] = None, **kwargs) → None¶

In keeping with data segment mode, write the data provided in raw form, assuming the slice specified relative to raw data coordinates. This requires that mode is ‘w’.

Only one of `start_indices` and `subscript` should be specified.

Parameters:

data (numpy.ndarray) – The data in raw form.
start_indices (None|int|Tuple[int, ...]) – Assuming a contiguous chunk of data, this provides the starting indices of the chunk. Any missing (tail) coordinates will be filled in with 0’s.
subscript (None|Tuple[slice, ...]) – The subscript definition in raw coordinates.
kwargs –

Return type:

None

check_fully_written(warn: bool = False) → bool¶

Checks that all expected pixel data is fully written.

Parameters:: warn (bool) – Log warning with some details if not fully written.
Return type:: bool

get_raw_bytes(warn: bool = True) → bytes | Tuple¶

This returns the bytes for the underlying raw data.

Warning

A data segment is conceptually represented in raw data as a single numpy array of appropriate shape and data type. When the data segment is formed from component pieces, then the return of this function may deviate significantly from the raw byte representation of such an array after consideration of data order and pad pixels.

Parameters:: warn (bool) – If True, then a check will be performed to ensure that the data has been fully written and warnings printed if the answer is no.
Returns:: The result will be a bytes object, unless the data segment is made up of a collection of child data segments, in which case the result will be a Tuple consisting of their get_raw_bytes returns.
Return type:: bytes|Tuple

flush() → None¶

Should perform, if possible, any necessary steps to flush any unwritten data to the file.

Return type:: None

close()¶: This should perform any necessary clean-up operations, like closing open file handles, deleting any temp files, etc

class sarpy.io.general.data_segment.ReorientationSegment(parent: DataSegment, formatted_dtype: str | dtype | None = None, formatted_shape: Tuple[int, ...] | None = None, reverse_axes: int | Sequence[int] | None = None, transpose_axes: Tuple[int, ...] | None = None, format_function: FormatFunction | None = None, close_parent: bool = True)¶

Bases: DataSegment

Define a basic ordering of a given DataSegment. The raw data will be presented as the parent data segment’s formatted data.

Introduced in version 1.3.0.

property close_parent: bool¶

Call parent.close() when close is called?

Type:: bool

In keeping with data segment mode, read raw data from the source, without reformatting and or applying symmetry operations. This requires that mode is ‘r’.

Parameters:

subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) – These arguments are relative to raw data shape and order, no symmetry operations have been applied.
squeeze (bool) – Apply numpy.squeeze, which eliminates any dimensions of size 1?

Returns:

This will be of data type given by raw_dtype.

Return type:

numpy.ndarray

check_fully_written(warn: bool = False) → bool¶

Checks that all expected pixel data is fully written.

Parameters:: warn (bool) – Log warning with some details if not fully written.
Return type:: bool

write_raw(data: ndarray, start_indices: None | int | Tuple[int, ...] = None, subscript: None | Sequence[slice] = None, **kwargs)¶

In keeping with data segment mode, write the data provided in raw form, assuming the slice specified relative to raw data coordinates. This requires that mode is ‘w’.

Note that raw form/order for the data segment is simply a reordered version of the formatted data for parent. This is not related to raw data with respect to the parent. To write raw data with respect to the parent, use parent.write_raw() instead.

Only one of `start_indices` and `subscript` should be specified.

Parameters:

data (numpy.ndarray) – The data in raw form.
start_indices (None|int|Tuple[int, ...]) – Assuming a contiguous chunk of data, this provides the starting indices of the chunk. Any missing (tail) coordinates will be filled in with 0’s.
subscript (None|Sequence[slice]) – The subscript definition in raw coordinates.
kwargs –

Return type:

None

get_raw_bytes(warn: bool = True) → bytes | Tuple¶

This returns the bytes for the underlying raw data.

Warning

A data segment is conceptually represented in raw data as a single numpy array of appropriate shape and data type. When the data segment is formed from component pieces, then the return of this function may deviate significantly from the raw byte representation of such an array after consideration of data order and pad pixels.

Parameters:: warn (bool) – If True, then a check will be performed to ensure that the data has been fully written and warnings printed if the answer is no.
Returns:: The result will be a bytes object, unless the data segment is made up of a collection of child data segments, in which case the result will be a Tuple consisting of their get_raw_bytes returns.
Return type:: bytes|Tuple

flush() → None¶

Should perform, if possible, any necessary steps to flush any unwritten data to the file.

Return type:: None

close()¶: This should perform any necessary clean-up operations, like closing open file handles, deleting any temp files, etc

property can_write_regular: bool¶

Can this data segment write regular data, which requires a function inverse?

Type:: bool

property closed: bool¶

Is the data segment closed? Reading or writing will result in a ValueError

Type:: bool

property format_function: FormatFunction¶

The format function which will be applied to the raw data.

Return type:: FormatFunction

property formatted_dtype: dtype¶

The data type of the formatted data, which will be returned by the read() function.

Type:: numpy.dtype

property formatted_ndim: int¶

The number of formatted dimensions.

Type:: int

property formatted_shape: Tuple[int, ...]¶

The formatted data shape.

Type:: Tuple[int, …]

property mode: str¶

The mode.

Type:: str

property raw_dtype: dtype¶

The data type of the data returned by the read_raw() function.

Type:: numpy.dtype

property raw_ndim: int¶

The number of raw dimensions.

Type:: int

property raw_shape: Tuple[int, ...]¶

The raw shape.

Type:: Tuple[int, …]

In keeping with data segment mode, read the data slice specified relative to the formatted data coordinates. This requires that mode is ‘r’.

Parameters:

subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
squeeze (bool) – Apply the numpy.squeeze operation, which eliminates dimension of size 1?

Return type:

numpy.ndarray

property reverse_axes: Tuple[int, ...] | None¶

The collection of axes (with respect to raw order) along which we will reverse as part of transformation to formatted data order. If not None, then this will be a tuple in strictly increasing order.

Type:: None|Tuple[int, …]

property transpose_axes: Tuple[int, ...]¶

The transpose order for switching from raw order to formatted order, prior to applying any format function.

If populated, this must be a permutation of (0, 1, …, raw_ndim-1).

Type:: None|Tuple[int, …]

Verifies that the structure of the subscript is in keeping with the formatted shape, and fills in any missing dimensions.

Parameters:: subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
Returns:: Guaranteed to be a tuple of slices of length formatted_ndim.
Return type:: Tuple[slice, …]

Verifies that the structure of the subscript is in keeping with the raw shape, and fills in any missing dimensions.

Parameters:: subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
Returns:: Guaranteed to be a tuple of slices of length raw_ndim.
Return type:: Tuple[slice, …]

write(data: ndarray, start_indices: None | int | Tuple[int, ...] = None, subscript: None | Sequence[slice] = None, **kwargs) → None¶

In keeping with data segment mode, write the data provided in formatted form, assuming the slice specified relative to the formatted data coordinates.

This requires that mode is ‘w’, and format_function.has_inverse == True, because we have to apply the format function inverse to the provided data.

Only one of `start_indices` and `subscript` should be specified.

Parameters:

data (numpy.ndarray) – The data in formatted form, to be transferred to raw form and written.
start_indices (None|int|Tuple[int, ...]) – Assuming a contiguous chunk of data, this provides the starting indices of the chunk. Any missing (tail) coordinates will be filled in with 0’s.
subscript (None|Sequence[slice]) – The subscript definition in formatted coordinates.
kwargs –

Return type:

None

class sarpy.io.general.data_segment.SubsetSegment(parent: DataSegment, subset_definition: Tuple[slice, ...], coordinate_basis: str, squeeze: bool = True, close_parent: bool = True)¶

Bases: DataSegment

Define a subset of a given DataSegment, with formatting handled by the parent data segment.

Introduced in version 1.3.0.

property formatted_subset_definition: Tuple[slice, ...]¶

The subset definition, in formatted coordinates.

Type:: Tuple[slice]

property raw_subset_definition: Tuple[slice, ...]¶

The subset definition, in raw coordinates.

Type:: Tuple[slice]

property close_parent: bool¶

Call parent.close() when close is called?

Type:: bool

get_parent_raw_subscript(subscript: None | int | slice | Sequence[int | slice]) → Tuple[slice, ...]¶

Gets the raw parent subscript from the raw subset subscript definition.

Parameters:: subscript (None|int|slice|Sequence[int|slice]) –
Return type:: Tuple[slice, …]

get_parent_formatted_subscript(subscript: None | int | slice | Sequence[int | slice]) → Tuple[slice, ...]¶

Gets the formatted parent subscript from the formatted subset subscript definition.

Parameters:: subscript (None|int|slice|Sequence[int|slice]) –
Return type:: Tuple[slice, …]

In keeping with data segment mode, read raw data from the source, without reformatting and or applying symmetry operations. This requires that mode is ‘r’.

Parameters:

subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) – These arguments are relative to raw data shape and order, no symmetry operations have been applied.
squeeze (bool) – Apply numpy.squeeze, which eliminates any dimensions of size 1?

Returns:

This will be of data type given by raw_dtype.

Return type:

numpy.ndarray

read(subscript: None | int | slice | Sequence[int | slice], squeeze=True) → ndarray¶

In keeping with data segment mode, read the data slice specified relative to the formatted data coordinates. This requires that mode is ‘r’.

Parameters:

subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
squeeze (bool) – Apply the numpy.squeeze operation, which eliminates dimension of size 1?

Return type:

numpy.ndarray

check_fully_written(warn: bool = False) → bool¶

Checks that all expected pixel data is fully written.

Parameters:: warn (bool) – Log warning with some details if not fully written.
Return type:: bool

write_raw(data: ndarray, start_indices: None | int | Tuple[int, ...] = None, subscript: None | Sequence[slice] = None, **kwargs)¶

In keeping with data segment mode, write the data provided in raw form, assuming the slice specified relative to raw data coordinates. This requires that mode is ‘w’.

Only one of `start_indices` and `subscript` should be specified.

Parameters:

data (numpy.ndarray) – The data in raw form.
start_indices (None|int|Tuple[int, ...]) – Assuming a contiguous chunk of data, this provides the starting indices of the chunk. Any missing (tail) coordinates will be filled in with 0’s.
subscript (None|Tuple[slice, ...]) – The subscript definition in raw coordinates.
kwargs –

Return type:

None

get_raw_bytes(warn: bool = True) → bytes | Tuple¶

This returns the bytes for the underlying raw data of the parent segment.

The only writing use case considered at present is for the blocks including padding inside a NITF file.

Parameters:: warn (bool) – If True, then a check will be performed to ensure that the data has been fully written.
Returns:: The result will be a bytes object, unless the data segment is made up of a collection of child data segments, in which case the result will be a Tuple consisting of their get_raw_bytes returns.
Return type:: bytes|Tuple

flush() → None¶

Should perform, if possible, any necessary steps to flush any unwritten data to the file.

Return type:: None

close()¶: This should perform any necessary clean-up operations, like closing open file handles, deleting any temp files, etc

property can_write_regular: bool¶

Can this data segment write regular data, which requires a function inverse?

Type:: bool

property closed: bool¶

Is the data segment closed? Reading or writing will result in a ValueError

Type:: bool

property format_function: FormatFunction¶

The format function which will be applied to the raw data.

Return type:: FormatFunction

property formatted_dtype: dtype¶

The data type of the formatted data, which will be returned by the read() function.

Type:: numpy.dtype

property formatted_ndim: int¶

The number of formatted dimensions.

Type:: int

property formatted_shape: Tuple[int, ...]¶

The formatted data shape.

Type:: Tuple[int, …]

property mode: str¶

The mode.

Type:: str

property raw_dtype: dtype¶

The data type of the data returned by the read_raw() function.

Type:: numpy.dtype

property raw_ndim: int¶

The number of raw dimensions.

Type:: int

property raw_shape: Tuple[int, ...]¶

The raw shape.

Type:: Tuple[int, …]

property reverse_axes: Tuple[int, ...] | None¶

The collection of axes (with respect to raw order) along which we will reverse as part of transformation to formatted data order. If not None, then this will be a tuple in strictly increasing order.

Type:: None|Tuple[int, …]

property transpose_axes: Tuple[int, ...]¶

The transpose order for switching from raw order to formatted order, prior to applying any format function.

If populated, this must be a permutation of (0, 1, …, raw_ndim-1).

Type:: None|Tuple[int, …]

Verifies that the structure of the subscript is in keeping with the formatted shape, and fills in any missing dimensions.

Parameters:: subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
Returns:: Guaranteed to be a tuple of slices of length formatted_ndim.
Return type:: Tuple[slice, …]

Verifies that the structure of the subscript is in keeping with the raw shape, and fills in any missing dimensions.

Parameters:: subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
Returns:: Guaranteed to be a tuple of slices of length raw_ndim.
Return type:: Tuple[slice, …]

write(data: ndarray, start_indices: None | int | Tuple[int, ...] = None, subscript: None | Sequence[slice] = None, **kwargs) → None¶

In keeping with data segment mode, write the data provided in formatted form, assuming the slice specified relative to the formatted data coordinates.

This requires that mode is ‘w’, and format_function.has_inverse == True, because we have to apply the format function inverse to the provided data.

Only one of `start_indices` and `subscript` should be specified.

Parameters:

data (numpy.ndarray) – The data in formatted form, to be transferred to raw form and written.
start_indices (None|int|Tuple[int, ...]) – Assuming a contiguous chunk of data, this provides the starting indices of the chunk. Any missing (tail) coordinates will be filled in with 0’s.
subscript (None|Sequence[slice]) – The subscript definition in formatted coordinates.
kwargs –

Return type:

None

class sarpy.io.general.data_segment.BandAggregateSegment(children: Sequence[DataSegment], band_dimension: int, formatted_dtype: str | dtype | None = None, formatted_shape: Tuple[int, ...] | None = None, reverse_axes: int | Sequence[int] | None = None, transpose_axes: Tuple[int, ...] | None = None, format_function: FormatFunction | None = None, close_children: bool = True)¶

Bases: DataSegment

This stacks a collection of data segments, which must have compatible details, together along a new (final) band dimension.

Note that read() and read_raw() return identical results here. To access raw data from the children, use access on the children property.

Introduced in version 1.3.0.

property band_dimension: int¶

The band dimension, in raw data after the transpose operation.

Type:: int

property close_children: bool¶

Call child.close() when close is called?

Type:: bool

property children: Tuple[DataSegment, ...]¶

The collection of children that we are stacking.

Return type:: Tuple[DataSegment, …]

property bands: int¶

The number of bands (child data segments)

Type:: int

In keeping with data segment mode, read raw data from the source, without reformatting and or applying symmetry operations. This requires that mode is ‘r’.

Parameters:

subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) – These arguments are relative to raw data shape and order, no symmetry operations have been applied.
squeeze (bool) – Apply numpy.squeeze, which eliminates any dimensions of size 1?

Returns:

This will be of data type given by raw_dtype.

Return type:

numpy.ndarray

check_fully_written(warn: bool = False) → bool¶

Checks that all expected pixel data is fully written.

Parameters:: warn (bool) – Log warning with some details if not fully written.
Return type:: bool

write_raw(data: ndarray, start_indices: None | int | Tuple[int, ...] = None, subscript: None | Sequence[slice] = None, **kwargs)¶

In keeping with data segment mode, write the data provided in raw form, assuming the slice specified relative to raw data coordinates. This requires that mode is ‘w’.

Note that raw form/order for the data segment is simply a reordered version of the formatted data for parent. This is not related to raw data with respect to the parent. To write raw data with respect to the parent, use parent.write_raw() instead.

Only one of `start_indices` and `subscript` should be specified.

Parameters:

data (numpy.ndarray) – The data in raw form.
start_indices (None|int|Tuple[int, ...]) – Assuming a contiguous chunk of data, this provides the starting indices of the chunk. Any missing (tail) coordinates will be filled in with 0’s.
subscript (None|Sequence[slice]) – The subscript definition in raw coordinates.
kwargs –

Return type:

None

get_raw_bytes(warn: bool = True) → bytes | Tuple¶

This returns the bytes for the underlying raw data.

Warning

A data segment is conceptually represented in raw data as a single numpy array of appropriate shape and data type. When the data segment is formed from component pieces, then the return of this function may deviate significantly from the raw byte representation of such an array after consideration of data order and pad pixels.

Parameters:: warn (bool) – If True, then a check will be performed to ensure that the data has been fully written and warnings printed if the answer is no.
Returns:: The result will be a bytes object, unless the data segment is made up of a collection of child data segments, in which case the result will be a Tuple consisting of their get_raw_bytes returns.
Return type:: bytes|Tuple

flush() → None¶

Should perform, if possible, any necessary steps to flush any unwritten data to the file.

Return type:: None

close()¶: This should perform any necessary clean-up operations, like closing open file handles, deleting any temp files, etc

property can_write_regular: bool¶

Can this data segment write regular data, which requires a function inverse?

Type:: bool

property closed: bool¶

Is the data segment closed? Reading or writing will result in a ValueError

Type:: bool

property format_function: FormatFunction¶

The format function which will be applied to the raw data.

Return type:: FormatFunction

property formatted_dtype: dtype¶

The data type of the formatted data, which will be returned by the read() function.

Type:: numpy.dtype

property formatted_ndim: int¶

The number of formatted dimensions.

Type:: int

property formatted_shape: Tuple[int, ...]¶

The formatted data shape.

Type:: Tuple[int, …]

property mode: str¶

The mode.

Type:: str

property raw_dtype: dtype¶

The data type of the data returned by the read_raw() function.

Type:: numpy.dtype

property raw_ndim: int¶

The number of raw dimensions.

Type:: int

property raw_shape: Tuple[int, ...]¶

The raw shape.

Type:: Tuple[int, …]

In keeping with data segment mode, read the data slice specified relative to the formatted data coordinates. This requires that mode is ‘r’.

Parameters:

subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
squeeze (bool) – Apply the numpy.squeeze operation, which eliminates dimension of size 1?

Return type:

numpy.ndarray

property reverse_axes: Tuple[int, ...] | None¶

The collection of axes (with respect to raw order) along which we will reverse as part of transformation to formatted data order. If not None, then this will be a tuple in strictly increasing order.

Type:: None|Tuple[int, …]

property transpose_axes: Tuple[int, ...]¶

The transpose order for switching from raw order to formatted order, prior to applying any format function.

If populated, this must be a permutation of (0, 1, …, raw_ndim-1).

Type:: None|Tuple[int, …]

Verifies that the structure of the subscript is in keeping with the formatted shape, and fills in any missing dimensions.

Parameters:: subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
Returns:: Guaranteed to be a tuple of slices of length formatted_ndim.
Return type:: Tuple[slice, …]

Verifies that the structure of the subscript is in keeping with the raw shape, and fills in any missing dimensions.

Parameters:: subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
Returns:: Guaranteed to be a tuple of slices of length raw_ndim.
Return type:: Tuple[slice, …]

write(data: ndarray, start_indices: None | int | Tuple[int, ...] = None, subscript: None | Sequence[slice] = None, **kwargs) → None¶

In keeping with data segment mode, write the data provided in formatted form, assuming the slice specified relative to the formatted data coordinates.

This requires that mode is ‘w’, and format_function.has_inverse == True, because we have to apply the format function inverse to the provided data.

Only one of `start_indices` and `subscript` should be specified.

Parameters:

data (numpy.ndarray) – The data in formatted form, to be transferred to raw form and written.
start_indices (None|int|Tuple[int, ...]) – Assuming a contiguous chunk of data, this provides the starting indices of the chunk. Any missing (tail) coordinates will be filled in with 0’s.
subscript (None|Sequence[slice]) – The subscript definition in formatted coordinates.
kwargs –

Return type:

None

class sarpy.io.general.data_segment.BlockAggregateSegment(children: Sequence[DataSegment], child_arrangement: Sequence[Tuple[slice, ...]], coordinate_basis: str, missing_data_value, raw_shape: Tuple[int, ...], formatted_dtype: str | dtype, formatted_shape: Tuple[int, ...], reverse_axes: int | Sequence[int] | None = None, transpose_axes: Tuple[int, ...] | None = None, format_function: FormatFunction | None = None, close_children: bool = True)¶

Bases: DataSegment

Combines a collection of child data segments, according to a given block definition. All children must have the same formatted_dtype. This implementation is motivated by a two-dimensional block arrangement, but is entirely general.

No effort is made to ensure that the block definition spans the whole space, nor is any effort made to ensure that blocks do not overlap.

If there are holes present in block definition, then data read across any hole will be populated with missing_data_value. Data attempted to write across any hole will simply be ignored.

Introduced in version 1.3.0.

property close_children: bool¶

Call child.close() when close is called?

Type:: bool

property children: Tuple[DataSegment, ...]¶

The collection of children that we are stacking together.

Return type:: Tuple[DataSegment, …]

In keeping with data segment mode, read raw data from the source, without reformatting and or applying symmetry operations. This requires that mode is ‘r’.

Parameters:

subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) – These arguments are relative to raw data shape and order, no symmetry operations have been applied.
squeeze (bool) – Apply numpy.squeeze, which eliminates any dimensions of size 1?

Returns:

This will be of data type given by raw_dtype.

Return type:

numpy.ndarray

check_fully_written(warn: bool = False) → bool¶

Checks that all expected pixel data is fully written.

Parameters:: warn (bool) – Log warning with some details if not fully written.
Return type:: bool

write_raw(data: ndarray, start_indices: None | int | Tuple[int, ...] = None, subscript: None | Sequence[slice] = None, **kwargs)¶

In keeping with data segment mode, write the data provided in raw form, assuming the slice specified relative to raw data coordinates. This requires that mode is ‘w’.

Note that raw form/order for the data segment is broken into blocks of reordered version of the formatted data for each child. This is not related to raw data with respect to the child. To write raw data with respect to the parent, use child.write_raw() instead.

Only one of `start_indices` and `subscript` should be specified.

Parameters:

data (numpy.ndarray) – The data in raw form.
start_indices (None|int|Tuple[int, ...]) – Assuming a contiguous chunk of data, this provides the starting indices of the chunk. Any missing (tail) coordinates will be filled in with 0’s.
subscript (None|Sequence[slice]) – The subscript definition in raw coordinates.
kwargs –

Return type:

None

get_raw_bytes(warn: bool = True) → bytes | Tuple¶

This returns the bytes for the underlying raw data.

Warning

A data segment is conceptually represented in raw data as a single numpy array of appropriate shape and data type. When the data segment is formed from component pieces, then the return of this function may deviate significantly from the raw byte representation of such an array after consideration of data order and pad pixels.

Parameters:: warn (bool) – If True, then a check will be performed to ensure that the data has been fully written and warnings printed if the answer is no.
Returns:: The result will be a bytes object, unless the data segment is made up of a collection of child data segments, in which case the result will be a Tuple consisting of their get_raw_bytes returns.
Return type:: bytes|Tuple

flush() → None¶

Should perform, if possible, any necessary steps to flush any unwritten data to the file.

Return type:: None

close()¶: This should perform any necessary clean-up operations, like closing open file handles, deleting any temp files, etc

property can_write_regular: bool¶

Can this data segment write regular data, which requires a function inverse?

Type:: bool

property closed: bool¶

Is the data segment closed? Reading or writing will result in a ValueError

Type:: bool

property format_function: FormatFunction¶

The format function which will be applied to the raw data.

Return type:: FormatFunction

property formatted_dtype: dtype¶

The data type of the formatted data, which will be returned by the read() function.

Type:: numpy.dtype

property formatted_ndim: int¶

The number of formatted dimensions.

Type:: int

property formatted_shape: Tuple[int, ...]¶

The formatted data shape.

Type:: Tuple[int, …]

property mode: str¶

The mode.

Type:: str

property raw_dtype: dtype¶

The data type of the data returned by the read_raw() function.

Type:: numpy.dtype

property raw_ndim: int¶

The number of raw dimensions.

Type:: int

property raw_shape: Tuple[int, ...]¶

The raw shape.

Type:: Tuple[int, …]

In keeping with data segment mode, read the data slice specified relative to the formatted data coordinates. This requires that mode is ‘r’.

Parameters:

subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
squeeze (bool) – Apply the numpy.squeeze operation, which eliminates dimension of size 1?

Return type:

numpy.ndarray

property reverse_axes: Tuple[int, ...] | None¶

The collection of axes (with respect to raw order) along which we will reverse as part of transformation to formatted data order. If not None, then this will be a tuple in strictly increasing order.

Type:: None|Tuple[int, …]

property transpose_axes: Tuple[int, ...]¶

The transpose order for switching from raw order to formatted order, prior to applying any format function.

If populated, this must be a permutation of (0, 1, …, raw_ndim-1).

Type:: None|Tuple[int, …]

Verifies that the structure of the subscript is in keeping with the formatted shape, and fills in any missing dimensions.

Parameters:: subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
Returns:: Guaranteed to be a tuple of slices of length formatted_ndim.
Return type:: Tuple[slice, …]

Verifies that the structure of the subscript is in keeping with the raw shape, and fills in any missing dimensions.

Parameters:: subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
Returns:: Guaranteed to be a tuple of slices of length raw_ndim.
Return type:: Tuple[slice, …]

write(data: ndarray, start_indices: None | int | Tuple[int, ...] = None, subscript: None | Sequence[slice] = None, **kwargs) → None¶

In keeping with data segment mode, write the data provided in formatted form, assuming the slice specified relative to the formatted data coordinates.

This requires that mode is ‘w’, and format_function.has_inverse == True, because we have to apply the format function inverse to the provided data.

Only one of `start_indices` and `subscript` should be specified.

Parameters:

data (numpy.ndarray) – The data in formatted form, to be transferred to raw form and written.
start_indices (None|int|Tuple[int, ...]) – Assuming a contiguous chunk of data, this provides the starting indices of the chunk. Any missing (tail) coordinates will be filled in with 0’s.
subscript (None|Sequence[slice]) – The subscript definition in formatted coordinates.
kwargs –

Return type:

None

class sarpy.io.general.data_segment.NumpyArraySegment(underlying_array: ndarray, formatted_dtype: str | dtype | None = None, formatted_shape: Tuple[int, ...] | None = None, reverse_axes: int | Sequence[int] | None = None, transpose_axes: Tuple[int, ...] | None = None, format_function: FormatFunction | None = None, mode: str = 'r')¶

Bases: DataSegment

DataSegment based on reading from a numpy.ndarray.

Introduced in version 1.3.0.

property underlying_array: ndarray¶

The underlying data array.

Return type:: numpy.ndarray

In keeping with data segment mode, read raw data from the source, without reformatting and or applying symmetry operations. This requires that mode is ‘r’.

Parameters:

subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) – These arguments are relative to raw data shape and order, no symmetry operations have been applied.
squeeze (bool) – Apply numpy.squeeze, which eliminates any dimensions of size 1?

Returns:

This will be of data type given by raw_dtype.

Return type:

numpy.ndarray

check_fully_written(warn: bool = False) → bool¶

Checks that all expected pixel data is fully written.

Parameters:: warn (bool) – Log warning with some details if not fully written.
Return type:: bool

write_raw(data: ndarray, start_indices: int | Tuple[int, ...] | None = None, subscript: Sequence[slice] | None = None, **kwargs)¶

In keeping with data segment mode, write the data provided in raw form, assuming the slice specified relative to raw data coordinates. This requires that mode is ‘w’.

Only one of `start_indices` and `subscript` should be specified.

Parameters:

data (numpy.ndarray) – The data in raw form.
start_indices (None|int|Tuple[int, ...]) – Assuming a contiguous chunk of data, this provides the starting indices of the chunk. Any missing (tail) coordinates will be filled in with 0’s.
subscript (None|Tuple[slice, ...]) – The subscript definition in raw coordinates.
kwargs –

Return type:

None

get_raw_bytes(warn: bool = False) → bytes | Tuple¶

This returns the bytes for the underlying raw data.

Warning

A data segment is conceptually represented in raw data as a single numpy array of appropriate shape and data type. When the data segment is formed from component pieces, then the return of this function may deviate significantly from the raw byte representation of such an array after consideration of data order and pad pixels.

Parameters:: warn (bool) – If True, then a check will be performed to ensure that the data has been fully written and warnings printed if the answer is no.
Returns:: The result will be a bytes object, unless the data segment is made up of a collection of child data segments, in which case the result will be a Tuple consisting of their get_raw_bytes returns.
Return type:: bytes|Tuple

flush() → None¶

Should perform, if possible, any necessary steps to flush any unwritten data to the file.

Return type:: None

close() → None¶: This should perform any necessary clean-up operations, like closing open file handles, deleting any temp files, etc

property can_write_regular: bool¶

Can this data segment write regular data, which requires a function inverse?

Type:: bool

property closed: bool¶

Is the data segment closed? Reading or writing will result in a ValueError

Type:: bool

property format_function: FormatFunction¶

The format function which will be applied to the raw data.

Return type:: FormatFunction

property formatted_dtype: dtype¶

The data type of the formatted data, which will be returned by the read() function.

Type:: numpy.dtype

property formatted_ndim: int¶

The number of formatted dimensions.

Type:: int

property formatted_shape: Tuple[int, ...]¶

The formatted data shape.

Type:: Tuple[int, …]

property mode: str¶

The mode.

Type:: str

property raw_dtype: dtype¶

The data type of the data returned by the read_raw() function.

Type:: numpy.dtype

property raw_ndim: int¶

The number of raw dimensions.

Type:: int

property raw_shape: Tuple[int, ...]¶

The raw shape.

Type:: Tuple[int, …]

In keeping with data segment mode, read the data slice specified relative to the formatted data coordinates. This requires that mode is ‘r’.

Parameters:

subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
squeeze (bool) – Apply the numpy.squeeze operation, which eliminates dimension of size 1?

Return type:

numpy.ndarray

property reverse_axes: Tuple[int, ...] | None¶

The collection of axes (with respect to raw order) along which we will reverse as part of transformation to formatted data order. If not None, then this will be a tuple in strictly increasing order.

Type:: None|Tuple[int, …]

property transpose_axes: Tuple[int, ...]¶

The transpose order for switching from raw order to formatted order, prior to applying any format function.

If populated, this must be a permutation of (0, 1, …, raw_ndim-1).

Type:: None|Tuple[int, …]

Verifies that the structure of the subscript is in keeping with the formatted shape, and fills in any missing dimensions.

Parameters:: subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
Returns:: Guaranteed to be a tuple of slices of length formatted_ndim.
Return type:: Tuple[slice, …]

Verifies that the structure of the subscript is in keeping with the raw shape, and fills in any missing dimensions.

Parameters:: subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
Returns:: Guaranteed to be a tuple of slices of length raw_ndim.
Return type:: Tuple[slice, …]

write(data: ndarray, start_indices: None | int | Tuple[int, ...] = None, subscript: None | Sequence[slice] = None, **kwargs) → None¶

In keeping with data segment mode, write the data provided in formatted form, assuming the slice specified relative to the formatted data coordinates.

This requires that mode is ‘w’, and format_function.has_inverse == True, because we have to apply the format function inverse to the provided data.

Only one of `start_indices` and `subscript` should be specified.

Parameters:

data (numpy.ndarray) – The data in formatted form, to be transferred to raw form and written.
start_indices (None|int|Tuple[int, ...]) – Assuming a contiguous chunk of data, this provides the starting indices of the chunk. Any missing (tail) coordinates will be filled in with 0’s.
subscript (None|Sequence[slice]) – The subscript definition in formatted coordinates.
kwargs –

Return type:

None

class sarpy.io.general.data_segment.NumpyMemmapSegment(file_object: str | BinaryIO, data_offset: int, raw_dtype: str | dtype, raw_shape: Tuple[int, ...], formatted_dtype: str | dtype | None = None, formatted_shape: Tuple[int, ...] | None = None, reverse_axes: int | Sequence[int] | None = None, transpose_axes: Tuple[int, ...] | None = None, format_function: FormatFunction | None = None, mode: str = 'r', close_file: bool = False)¶

Bases: NumpyArraySegment

DataSegment based on establishing a numpy memmap, and using that as the underlying array.

Introduced in version 1.3.0.

property close_file: bool¶

Close the file object when complete?

Type:: bool

flush() → None¶

Should perform, if possible, any necessary steps to flush any unwritten data to the file.

Return type:: None

close()¶: This should perform any necessary clean-up operations, like closing open file handles, deleting any temp files, etc

property can_write_regular: bool¶

Can this data segment write regular data, which requires a function inverse?

Type:: bool

check_fully_written(warn: bool = False) → bool¶

Checks that all expected pixel data is fully written.

Parameters:: warn (bool) – Log warning with some details if not fully written.
Return type:: bool

property closed: bool¶

Is the data segment closed? Reading or writing will result in a ValueError

Type:: bool

property format_function: FormatFunction¶

The format function which will be applied to the raw data.

Return type:: FormatFunction

property formatted_dtype: dtype¶

The data type of the formatted data, which will be returned by the read() function.

Type:: numpy.dtype

property formatted_ndim: int¶

The number of formatted dimensions.

Type:: int

property formatted_shape: Tuple[int, ...]¶

The formatted data shape.

Type:: Tuple[int, …]

get_raw_bytes(warn: bool = False) → bytes | Tuple¶

This returns the bytes for the underlying raw data.

Warning

A data segment is conceptually represented in raw data as a single numpy array of appropriate shape and data type. When the data segment is formed from component pieces, then the return of this function may deviate significantly from the raw byte representation of such an array after consideration of data order and pad pixels.

Parameters:: warn (bool) – If True, then a check will be performed to ensure that the data has been fully written and warnings printed if the answer is no.
Returns:: The result will be a bytes object, unless the data segment is made up of a collection of child data segments, in which case the result will be a Tuple consisting of their get_raw_bytes returns.
Return type:: bytes|Tuple

property mode: str¶

The mode.

Type:: str

property raw_dtype: dtype¶

The data type of the data returned by the read_raw() function.

Type:: numpy.dtype

property raw_ndim: int¶

The number of raw dimensions.

Type:: int

property raw_shape: Tuple[int, ...]¶

The raw shape.

Type:: Tuple[int, …]

In keeping with data segment mode, read the data slice specified relative to the formatted data coordinates. This requires that mode is ‘r’.

Parameters:

subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
squeeze (bool) – Apply the numpy.squeeze operation, which eliminates dimension of size 1?

Return type:

numpy.ndarray

In keeping with data segment mode, read raw data from the source, without reformatting and or applying symmetry operations. This requires that mode is ‘r’.

Parameters:

subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) – These arguments are relative to raw data shape and order, no symmetry operations have been applied.
squeeze (bool) – Apply numpy.squeeze, which eliminates any dimensions of size 1?

Returns:

This will be of data type given by raw_dtype.

Return type:

numpy.ndarray

property reverse_axes: Tuple[int, ...] | None¶

The collection of axes (with respect to raw order) along which we will reverse as part of transformation to formatted data order. If not None, then this will be a tuple in strictly increasing order.

Type:: None|Tuple[int, …]

property transpose_axes: Tuple[int, ...]¶

The transpose order for switching from raw order to formatted order, prior to applying any format function.

If populated, this must be a permutation of (0, 1, …, raw_ndim-1).

Type:: None|Tuple[int, …]

property underlying_array: ndarray¶

The underlying data array.

Return type:: numpy.ndarray

Verifies that the structure of the subscript is in keeping with the formatted shape, and fills in any missing dimensions.

Parameters:: subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
Returns:: Guaranteed to be a tuple of slices of length formatted_ndim.
Return type:: Tuple[slice, …]

Verifies that the structure of the subscript is in keeping with the raw shape, and fills in any missing dimensions.

Parameters:: subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
Returns:: Guaranteed to be a tuple of slices of length raw_ndim.
Return type:: Tuple[slice, …]

write(data: ndarray, start_indices: None | int | Tuple[int, ...] = None, subscript: None | Sequence[slice] = None, **kwargs) → None¶

In keeping with data segment mode, write the data provided in formatted form, assuming the slice specified relative to the formatted data coordinates.

This requires that mode is ‘w’, and format_function.has_inverse == True, because we have to apply the format function inverse to the provided data.

Only one of `start_indices` and `subscript` should be specified.

Parameters:

data (numpy.ndarray) – The data in formatted form, to be transferred to raw form and written.
start_indices (None|int|Tuple[int, ...]) – Assuming a contiguous chunk of data, this provides the starting indices of the chunk. Any missing (tail) coordinates will be filled in with 0’s.
subscript (None|Sequence[slice]) – The subscript definition in formatted coordinates.
kwargs –

Return type:

None

write_raw(data: ndarray, start_indices: int | Tuple[int, ...] | None = None, subscript: Sequence[slice] | None = None, **kwargs)¶

In keeping with data segment mode, write the data provided in raw form, assuming the slice specified relative to raw data coordinates. This requires that mode is ‘w’.

Only one of `start_indices` and `subscript` should be specified.

Parameters:

data (numpy.ndarray) – The data in raw form.
start_indices (None|int|Tuple[int, ...]) – Assuming a contiguous chunk of data, this provides the starting indices of the chunk. Any missing (tail) coordinates will be filled in with 0’s.
subscript (None|Tuple[slice, ...]) – The subscript definition in raw coordinates.
kwargs –

Return type:

None

Bases: DataSegment

DataSegment based on reading from an hdf5 file, using the h5py library.

Introduced in version 1.3.0.

property close_file: bool¶

Close the file object when complete?

Type:: bool

In keeping with data segment mode, read raw data from the source, without reformatting and or applying symmetry operations. This requires that mode is ‘r’.

Parameters:

subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) – These arguments are relative to raw data shape and order, no symmetry operations have been applied.
squeeze (bool) – Apply numpy.squeeze, which eliminates any dimensions of size 1?

Returns:

This will be of data type given by raw_dtype.

Return type:

numpy.ndarray

write_raw(data: ndarray, start_indices: None | int | Tuple[int, ...] = None, subscript: None | Sequence[slice] = None, **kwargs)¶

In keeping with data segment mode, write the data provided in raw form, assuming the slice specified relative to raw data coordinates. This requires that mode is ‘w’.

Only one of `start_indices` and `subscript` should be specified.

Parameters:

data (numpy.ndarray) – The data in raw form.
start_indices (None|int|Tuple[int, ...]) – Assuming a contiguous chunk of data, this provides the starting indices of the chunk. Any missing (tail) coordinates will be filled in with 0’s.
subscript (None|Tuple[slice, ...]) – The subscript definition in raw coordinates.
kwargs –

Return type:

None

get_raw_bytes(warn: bool = True) → bytes | Tuple¶

This returns the bytes for the underlying raw data.

Warning

A data segment is conceptually represented in raw data as a single numpy array of appropriate shape and data type. When the data segment is formed from component pieces, then the return of this function may deviate significantly from the raw byte representation of such an array after consideration of data order and pad pixels.

Parameters:: warn (bool) – If True, then a check will be performed to ensure that the data has been fully written and warnings printed if the answer is no.
Returns:: The result will be a bytes object, unless the data segment is made up of a collection of child data segments, in which case the result will be a Tuple consisting of their get_raw_bytes returns.
Return type:: bytes|Tuple

check_fully_written(warn: bool = False) → bool¶

Checks that all expected pixel data is fully written.

Parameters:: warn (bool) – Log warning with some details if not fully written.
Return type:: bool

close() → None¶: This should perform any necessary clean-up operations, like closing open file handles, deleting any temp files, etc

property can_write_regular: bool¶

Can this data segment write regular data, which requires a function inverse?

Type:: bool

property closed: bool¶

Is the data segment closed? Reading or writing will result in a ValueError

Type:: bool

flush() → None¶

Should perform, if possible, any necessary steps to flush any unwritten data to the file.

Return type:: None

property format_function: FormatFunction¶

The format function which will be applied to the raw data.

Return type:: FormatFunction

property formatted_dtype: dtype¶

The data type of the formatted data, which will be returned by the read() function.

Type:: numpy.dtype

property formatted_ndim: int¶

The number of formatted dimensions.

Type:: int

property formatted_shape: Tuple[int, ...]¶

The formatted data shape.

Type:: Tuple[int, …]

property mode: str¶

The mode.

Type:: str

property raw_dtype: dtype¶

The data type of the data returned by the read_raw() function.

Type:: numpy.dtype

property raw_ndim: int¶

The number of raw dimensions.

Type:: int

property raw_shape: Tuple[int, ...]¶

The raw shape.

Type:: Tuple[int, …]

In keeping with data segment mode, read the data slice specified relative to the formatted data coordinates. This requires that mode is ‘r’.

Parameters:

subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
squeeze (bool) – Apply the numpy.squeeze operation, which eliminates dimension of size 1?

Return type:

numpy.ndarray

property reverse_axes: Tuple[int, ...] | None¶

The collection of axes (with respect to raw order) along which we will reverse as part of transformation to formatted data order. If not None, then this will be a tuple in strictly increasing order.

Type:: None|Tuple[int, …]

property transpose_axes: Tuple[int, ...]¶

The transpose order for switching from raw order to formatted order, prior to applying any format function.

If populated, this must be a permutation of (0, 1, …, raw_ndim-1).

Type:: None|Tuple[int, …]

Verifies that the structure of the subscript is in keeping with the formatted shape, and fills in any missing dimensions.

Parameters:: subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
Returns:: Guaranteed to be a tuple of slices of length formatted_ndim.
Return type:: Tuple[slice, …]

Verifies that the structure of the subscript is in keeping with the raw shape, and fills in any missing dimensions.

Parameters:: subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
Returns:: Guaranteed to be a tuple of slices of length raw_ndim.
Return type:: Tuple[slice, …]

write(data: ndarray, start_indices: None | int | Tuple[int, ...] = None, subscript: None | Sequence[slice] = None, **kwargs) → None¶

In keeping with data segment mode, write the data provided in formatted form, assuming the slice specified relative to the formatted data coordinates.

This requires that mode is ‘w’, and format_function.has_inverse == True, because we have to apply the format function inverse to the provided data.

Only one of `start_indices` and `subscript` should be specified.

Parameters:

data (numpy.ndarray) – The data in formatted form, to be transferred to raw form and written.
start_indices (None|int|Tuple[int, ...]) – Assuming a contiguous chunk of data, this provides the starting indices of the chunk. Any missing (tail) coordinates will be filled in with 0’s.
subscript (None|Sequence[slice]) – The subscript definition in formatted coordinates.
kwargs –

Return type:

None

class sarpy.io.general.data_segment.FileReadDataSegment(file_object: BinaryIO, data_offset: int, raw_dtype: str | dtype, raw_shape: Tuple[int, ...], formatted_dtype: str | dtype, formatted_shape: Tuple[int, ...], reverse_axes: int | Sequence[int] | None = None, transpose_axes: Tuple[int, ...] | None = None, format_function: FormatFunction | None = None, close_file: bool = False)¶

Bases: DataSegment

Read a data array manually from a file - this is primarily intended for cloud usage.

Introduced in version 1.3.0.

property can_write_regular: bool¶

Can this data segment write regular data, which requires a function inverse?

Type:: bool

property closed: bool¶

Is the data segment closed? Reading or writing will result in a ValueError

Type:: bool

flush() → None¶

Should perform, if possible, any necessary steps to flush any unwritten data to the file.

Return type:: None

property format_function: FormatFunction¶

The format function which will be applied to the raw data.

Return type:: FormatFunction

property formatted_dtype: dtype¶

The data type of the formatted data, which will be returned by the read() function.

Type:: numpy.dtype

property formatted_ndim: int¶

The number of formatted dimensions.

Type:: int

property formatted_shape: Tuple[int, ...]¶

The formatted data shape.

Type:: Tuple[int, …]

property mode: str¶

The mode.

Type:: str

property raw_dtype: dtype¶

The data type of the data returned by the read_raw() function.

Type:: numpy.dtype

property raw_ndim: int¶

The number of raw dimensions.

Type:: int

property raw_shape: Tuple[int, ...]¶

The raw shape.

Type:: Tuple[int, …]

In keeping with data segment mode, read the data slice specified relative to the formatted data coordinates. This requires that mode is ‘r’.

Parameters:

subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
squeeze (bool) – Apply the numpy.squeeze operation, which eliminates dimension of size 1?

Return type:

numpy.ndarray

property reverse_axes: Tuple[int, ...] | None¶

The collection of axes (with respect to raw order) along which we will reverse as part of transformation to formatted data order. If not None, then this will be a tuple in strictly increasing order.

Type:: None|Tuple[int, …]

property transpose_axes: Tuple[int, ...]¶

The transpose order for switching from raw order to formatted order, prior to applying any format function.

If populated, this must be a permutation of (0, 1, …, raw_ndim-1).

Type:: None|Tuple[int, …]

Verifies that the structure of the subscript is in keeping with the formatted shape, and fills in any missing dimensions.

Parameters:: subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
Returns:: Guaranteed to be a tuple of slices of length formatted_ndim.
Return type:: Tuple[slice, …]

Verifies that the structure of the subscript is in keeping with the raw shape, and fills in any missing dimensions.

Parameters:: subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) –
Returns:: Guaranteed to be a tuple of slices of length raw_ndim.
Return type:: Tuple[slice, …]

write(data: ndarray, start_indices: None | int | Tuple[int, ...] = None, subscript: None | Sequence[slice] = None, **kwargs) → None¶

In keeping with data segment mode, write the data provided in formatted form, assuming the slice specified relative to the formatted data coordinates.

This requires that mode is ‘w’, and format_function.has_inverse == True, because we have to apply the format function inverse to the provided data.

Only one of `start_indices` and `subscript` should be specified.

Parameters:

data (numpy.ndarray) – The data in formatted form, to be transferred to raw form and written.
start_indices (None|int|Tuple[int, ...]) – Assuming a contiguous chunk of data, this provides the starting indices of the chunk. Any missing (tail) coordinates will be filled in with 0’s.
subscript (None|Sequence[slice]) – The subscript definition in formatted coordinates.
kwargs –

Return type:

None

property close_file: bool¶

Close the file object when complete?

Type:: bool

property data_offset: int¶

The offset of the data in bytes from the start of the file-like object.

Type:: int

In keeping with data segment mode, read raw data from the source, without reformatting and or applying symmetry operations. This requires that mode is ‘r’.

Parameters:

subscript (None|int|slice|Sequence[int|slice|Tuple[int, ...]]) – These arguments are relative to raw data shape and order, no symmetry operations have been applied.
squeeze (bool) – Apply numpy.squeeze, which eliminates any dimensions of size 1?

Returns:

This will be of data type given by raw_dtype.

Return type:

numpy.ndarray

write_raw(data: ndarray, start_indices: None | int | Tuple[int, ...] = None, subscript: None | Sequence[slice] = None, **kwargs)¶

In keeping with data segment mode, write the data provided in raw form, assuming the slice specified relative to raw data coordinates. This requires that mode is ‘w’.

Only one of `start_indices` and `subscript` should be specified.

Parameters:

data (numpy.ndarray) – The data in raw form.
start_indices (None|int|Tuple[int, ...]) – Assuming a contiguous chunk of data, this provides the starting indices of the chunk. Any missing (tail) coordinates will be filled in with 0’s.
subscript (None|Tuple[slice, ...]) – The subscript definition in raw coordinates.
kwargs –

Return type:

None

get_raw_bytes(warn: bool = True) → bytes | Tuple¶

This returns the bytes for the underlying raw data.

Warning

A data segment is conceptually represented in raw data as a single numpy array of appropriate shape and data type. When the data segment is formed from component pieces, then the return of this function may deviate significantly from the raw byte representation of such an array after consideration of data order and pad pixels.

Parameters:: warn (bool) – If True, then a check will be performed to ensure that the data has been fully written and warnings printed if the answer is no.
Returns:: The result will be a bytes object, unless the data segment is made up of a collection of child data segments, in which case the result will be a Tuple consisting of their get_raw_bytes returns.
Return type:: bytes|Tuple

check_fully_written(warn: bool = False) → bool¶

Checks that all expected pixel data is fully written.

Parameters:: warn (bool) – Log warning with some details if not fully written.
Return type:: bool

close() → None¶: This should perform any necessary clean-up operations, like closing open file handles, deleting any temp files, etc

Data segment definition (sarpy.io.general.data_segment)¶

Table of Contents

Previous topic

Next topic

This Page