Class VisualFrame

Class Documentation

class aslam::VisualFrame

An image and keypoints from a single camera.

This class stores data from an image and keypoints taken from a single camera. It stores a pointer to the camera’s intrinsic calibration, an id that uniquely identifies this frame, and a measurement timestamp.

The class also stores a ChannelGroup object that can be used to hold keypoint data, the raw image, and other associated information.

The camera geometry stored in the frame and accessible by getCameraGeometry() may refer to a transformed geometry that includes downsampling and undistortion. However, we recommend to always store the raw image with the frame so that no information is lost. In order to be able to plot on these raw images, we must also store the original camera geometry object (accessible with getRawCameraGeometry()). To plot transformed keypoints on the raw image, one may use toRawImageCoordinates() or toRawImageCoordinatesVectorized() to recover the raw image coordinates of any keypoint.

Constructors/destructors and operators

VisualFrame()
inline virtual ~VisualFrame()
VisualFrame(const VisualFrame &other)

Copy constructor for clone operation. (Cameras are not cloned!)

VisualFrame &operator=(const VisualFrame &other)
virtual bool operator==(const VisualFrame &other) const

Public Types

typedef Eigen::Matrix<unsigned char, Eigen::Dynamic, Eigen::Dynamic> DescriptorsT

The descriptor matrix stores descriptors in columns, i.e.

the descriptor matrix has num_bytes_per_descriptor rows and num_descriptors columns.

typedef Eigen::VectorXd KeypointScoresT

Public Functions

ASLAM_POINTER_TYPEDEFS(VisualFrame)
virtual bool compareWithoutCameraGeometry(const VisualFrame &other) const
template<typename CHANNEL_DATA_TYPE>
inline void addChannel(const std::string &channel)
bool hasKeypointMeasurements() const

Are there keypoint measurements stored in this frame?

bool hasKeypointMeasurementUncertainties() const

Are there keypoint measurement uncertainties stored in this frame?

bool hasKeypointOrientations() const

Are there keypoint orientations stored in this frame?

bool hasKeypointScores() const

Are there keypoint scores stored in this frame?

bool hasKeypointScales() const

Are there keypoint scales stored in this frame?

bool hasDescriptors() const

Are there descriptors stored in this frame?

bool hasTrackIds() const

Are there track ids stored in this frame?

bool hasRawImage() const

Is there a raw image stored in this frame?

inline bool hasChannel(const std::string &channel) const

Is a certain channel stored in this frame?

void clearKeypointChannels()

Clears the following channels: KeypointMeasurements, KeypointMeasurementUncertainties, KeypointOrientations, KeypointScores, KeypointScales, Descriptors, TrackIds.

const Eigen::Matrix2Xd &getKeypointMeasurements() const

The keypoint measurements stored in a frame.

inline size_t getNumKeypointMeasurements() const

Get the number of keypoints measurements stored in this frame.

const Eigen::VectorXd &getKeypointMeasurementUncertainties() const

The keypoint measurement uncertainties stored in a frame.

const Eigen::VectorXd &getKeypointOrientations() const

The keypoint orientations stored in a frame.

const Eigen::VectorXd &getKeypointScores() const

The keypoint scores stored in a frame.

const Eigen::VectorXd &getKeypointScales() const

The keypoint scales stored in a frame.

const DescriptorsT &getDescriptors() const

The descriptors stored in a frame.

const Eigen::VectorXi &getTrackIds() const

The track ids stored in this frame.

const cv::Mat &getRawImage() const

The raw image stored in a frame.

void releaseRawImage()

Release the raw image. Only if the cv::Mat reference count is 1 the memory will be freed.

template<typename CHANNEL_DATA_TYPE>
inline const CHANNEL_DATA_TYPE &getChannelData(const std::string &channel) const
Eigen::Matrix2Xd *getKeypointMeasurementsMutable()

A pointer to the keypoint measurements, can be used to swap in new data.

Eigen::VectorXd *getKeypointMeasurementUncertaintiesMutable()

A pointer to the keypoint measurement uncertainties, can be used to swap in new data.

Eigen::VectorXd *getKeypointOrientationsMutable()

A pointer to the keypoint orientations, can be used to swap in new data.

Eigen::VectorXd *getKeypointScoresMutable()

A pointer to the keypoint scores, can be used to swap in new data.

Eigen::VectorXd *getKeypointScalesMutable()

A pointer to the keypoint scales, can be used to swap in new data.

DescriptorsT *getDescriptorsMutable()

A pointer to the descriptors, can be used to swap in new data.

Eigen::VectorXi *getTrackIdsMutable()

A pointer to the track ids, can be used to swap in new data.

cv::Mat *getRawImageMutable()

A pointer to the raw image, can be used to swap in new data.

template<typename CHANNEL_DATA_TYPE>
inline CHANNEL_DATA_TYPE *getChannelDataMutable(const std::string &channel) const
const Eigen::Block<Eigen::Matrix2Xd, 2, 1> getKeypointMeasurement(size_t index) const

Return block expression of the keypoint measurement pointed to by index.

double getKeypointMeasurementUncertainty(size_t index) const

Return the keypoint measurement uncertainty at index.

double getKeypointOrientation(size_t index) const

Return the keypoint orientation at index.

double getKeypointScore(size_t index) const

Return the keypoint score at index.

double getKeypointScale(size_t index) const

Return the keypoint scale at index.

const unsigned char *getDescriptor(size_t index) const

Return pointer location of the descriptor pointed to by index.

int getTrackId(size_t index) const

Return the track id at index. (-1: not tracked)

void setKeypointMeasurements(const Eigen::Matrix2Xd &keypoints)

Replace (copy) the internal keypoint measurements by the passed ones.

void setKeypointMeasurementUncertainties(const Eigen::VectorXd &uncertainties)

Replace (copy) the internal keypoint measurement uncertainties by the passed ones.

void setKeypointOrientations(const Eigen::VectorXd &orientations)

Replace (copy) the internal keypoint orientations by the passed ones.

void setKeypointScores(const Eigen::VectorXd &scores)

Replace (copy) the internal keypoint scores by the passed ones.

void setKeypointScales(const Eigen::VectorXd &scales)

Replace (copy) the internal keypoint orientations by the passed ones.

void setDescriptors(const DescriptorsT &descriptors)

Replace (copy) the internal descriptors by the passed ones.

void setDescriptors(const Eigen::Map<const DescriptorsT> &descriptors)

Replace (copy) the internal descriptors by the passed ones.

void setTrackIds(const Eigen::VectorXi &track_ids)

Replace (copy) the internal track ids by the passed ones.

void setRawImage(const cv::Mat &image)

Replace (copy) the internal raw image by the passed ones.

This is a shallow copy by default. Please clone the image if it should be owned by the VisualFrame.

template<typename CHANNEL_DATA_TYPE>
inline void setChannelData(const std::string &channel, const CHANNEL_DATA_TYPE &data_new)
void swapKeypointMeasurements(Eigen::Matrix2Xd *keypoints)

Replace (swap) the internal keypoint measurements by the passed ones.

This method creates the channel if it doesn’t exist

void swapKeypointMeasurementUncertainties(Eigen::VectorXd *uncertainties)

Replace (swap) the internal keypoint measurement uncertainties by the passed ones.

void swapKeypointOrientations(Eigen::VectorXd *orientations)

Replace (swap) the internal keypoint orientations by the passed ones.

void swapKeypointScores(Eigen::VectorXd *scores)

Replace (swap) the internal keypoint scores by the passed ones.

void swapKeypointScales(Eigen::VectorXd *scales)

Replace (swap) the internal keypoint orientations by the passed ones.

void swapDescriptors(DescriptorsT *descriptors)

Replace (swap) the internal descriptors by the passed ones.

void swapTrackIds(Eigen::VectorXi *track_ids)

Replace (swap) the internal track ids by the passed ones.

template<typename CHANNEL_DATA_TYPE>
inline void swapChannelData(const std::string &channel, CHANNEL_DATA_TYPE *data_new)

Swap channel data with the data passed in.

This will only work if the channel data type has a swap() method.

const Camera::ConstPtr getCameraGeometry() const

The camera geometry.

void setCameraGeometry(const Camera::ConstPtr &camera)

Set the camera geometry.

const Camera::ConstPtr getRawCameraGeometry() const

The camera geometry.

void setRawCameraGeometry(const Camera::ConstPtr &camera)

Set the camera geometry.

aslam::ProjectionResult getKeypointInRawImageCoordinates(size_t keypoint_idx, Eigen::Vector2d *keypoint_raw_coordinates) const

Return keypoint measurement pointed to by index in raw image coordinates.

Parameters
  • keypoint_idx[in] Index of the keypoint.

  • keypoint_raw_coordinates[out] The keypoint in raw image coordinates.

Returns

Was the projection successful?

aslam::ProjectionResult toRawImageCoordinates(const Eigen::Vector2d &keypoint, Eigen::Vector2d *out_image_coordinates) const

Convert keypoint coordinates to raw image coordinates.

Parameters
  • keypoint[in] The keypoint coordinates with respect to the camera calibration available from getCameraGeometry().

  • out_image_coordinates[out] The image coordinates with respect to the camera calibration available from getRawCameraGeometry().

void toRawImageCoordinatesVectorized(const Eigen::Matrix2Xd &keypoints, Eigen::Matrix2Xd *out_image_coordinates, std::vector<aslam::ProjectionResult> *results) const

Convert keypoint coordinates to raw image coordinates.

Parameters
  • keypoints[in] The keypoint coordinates with respect to the camera calibration available from getCameraGeometry().

  • out_image_coordinates[out] The image coordinates with respect to the camera calibration available from getRawCameraGeometry().

  • results[out] One result for each keypoint.

Eigen::Matrix3Xd getNormalizedBearingVectors(const std::vector<size_t> &keypoint_indices, std::vector<unsigned char> *backprojection_success) const

Return a list of normalized bearing vectors for the specified keypoint indices.

inline const aslam::FrameId &getId() const

Get the frame id.

inline void setId(const aslam::FrameId &id)

Set the frame id.

inline int64_t getTimestampNanoseconds() const

Get the timestamp.

inline void setTimestampNanoseconds(int64_t timestamp_nanoseconds)

Set the timestamp.

size_t getDescriptorSizeBytes() const

Set the size of the descriptor in bytes.

inline void validate()

Set the validity flag to true.

inline void invalidate()

Set the validity flag to false.

inline bool isValid() const

Check the validity flag.

inline void setValid(bool is_valid)

Set the validity flag.

void print(std::ostream &out, const std::string &label) const

Print out a human-readable version of this frame.

void discardUntrackedObservations(std::vector<size_t> *discarded_indices)

Public Static Functions

static VisualFrame::Ptr createEmptyTestVisualFrame(const aslam::Camera::ConstPtr &camera, int64_t timestamp_nanoseconds)

Creates an empty frame.

The following channels are added without any data attached: {KeypointMeasurements, KeypointMeasurementUncertainties, Descriptors}

Parameters
  • camera[in] Camera which will be assigned to the frame.

  • timestamp_nanoseconds[in] Timestamp of the frame. [ns]

Returns

Pointer to the created frame.