test5.AprilTagPianoDetector Class Reference

ros 2 node combining apriltag detection with yolo based piano key segmenytation on each incoming image the node: 1) Runs Yolo segmengation and detection to locate piano keys 2) Detects AprilTags 3)Anotate a previw frame and show it in an opencv window. More...

Inheritance diagram for test5.AprilTagPianoDetector:

Collaboration diagram for test5.AprilTagPianoDetector:

Public Member Functions
	__init__ (self)
	image_callback (self, Image msg)
	Ros 2 subscriber calback prosses one incoming colour image perfore mthe full detection, annotation and the publish pipeline: 1) convert the Ros image massages to a RGB Numpy array.

Public Attributes
	subscription
	image_callback
	publisher_tag
	publisher_keys
	device
	yolo_model
	tag_detector
	bridge
	ordered_keys

Protected Member Functions
list	_detect_piano_keys (self, np.ndarray bgr_image)
None	_draw_piano_keys (self, np.ndarray bgr_image, list keys)
	Runs yolo interfece on a RGB frame and returns ordere key metdata.
None	_publish_keys (self)
	publishing the current ordered_keys list as a poseArray message.
dict|None	_find_nearest_key (self, float px, float py)
	returns the key whose adjusted centroid is closest to a query point.

Protected Attributes
	_frame_count
	_fps_t0
	_fps_count
	_fps

Detailed Description

ros 2 node combining apriltag detection with yolo based piano key segmenytation on each incoming image the node: 1) Runs Yolo segmengation and detection to locate piano keys 2) Detects AprilTags 3)Anotate a previw frame and show it in an opencv window.

4) publish key poses andtag offsets to downstream node

Constructor & Destructor Documentation

__init__()

test5.AprilTagPianoDetector.__init__

(

self

)

Here is the call graph for this function:

Here is the caller graph for this function:

Member Function Documentation

_detect_piano_keys()

list test5.AprilTagPianoDetector._detect_piano_keys (   self, np.ndarray  bgr_image  )

protected

Here is the caller graph for this function:

_draw_piano_keys()

None test5.AprilTagPianoDetector._draw_piano_keys (   self, np.ndarray  bgr_image, list  keys  )

protected

Runs yolo interfece on a RGB frame and returns ordere key metdata.

when the model returns segmentation masks the centroid is computed from the mask poligon moments. otherwise the bounding box center is used.,

keys are sorted left to right by theyer boaunding box x1 cordinate and assigned a sequance key index.

Each return dict has the following fields:, 1) key_index: left to right ordring 2) label: class name from the yolo modle. 3) bbox: [x1, y1, x2, y2] in pixel cordinats 4) contour: Mask polygon points or [] if avalibal 5) mid_x and mid_y: Adjused centroid of x and y after applying the offset 6) confidance: yolo detection confidance score.

Parameters

bgr_image

Full resolution BGR frame as a Numpy uint8 array.

Returns

List of key dicts sorted by horizontal position

Here is the caller graph for this function:

_find_nearest_key()

dict | None test5.AprilTagPianoDetector._find_nearest_key (   self, float  px, float  py  )

protected

returns the key whose adjusted centroid is closest to a query point.

Uses squared euclidean distance with this no sqrt required.

Parameters

px	Query point x cordinate in pixel space.
py	Query point y cordinate in pixel space.

Returns

: The nearest key dict or none if no keys have been detected

Here is the caller graph for this function:

_publish_keys()

None test5.AprilTagPianoDetector._publish_keys (   self )

protected

publishing the current ordered_keys list as a poseArray message.

each key is encoded as a geometry_msgs/pose where: 1) pose.position.x: adjusted cordinat of x. 2) pose.position.y: adjusted cordinat of y. 3) pose.position.z: 0.0

poses are ordered left to right maching key-index in the source list

Here is the caller graph for this function:

image_callback()

test5.AprilTagPianoDetector.image_callback	(		self,
		Image	msg
	)

Ros 2 subscriber calback prosses one incoming colour image perfore mthe full detection, annotation and the publish pipeline: 1) convert the Ros image massages to a RGB Numpy array.

2)runs Apriltag detection on a greyscale copy. 3) annotates and displayes the preview frame 4)publishes tag offsets and key positions. 5)shuts the node down cleany when the user presses q

param msg sensor_msgs/image massages from the camera topic.

Here is the call graph for this function:

Member Data Documentation

_fps

test5.AprilTagPianoDetector._fps

protected

_fps_count

test5.AprilTagPianoDetector._fps_count

protected

_fps_t0

test5.AprilTagPianoDetector._fps_t0

protected

_frame_count

test5.AprilTagPianoDetector._frame_count

protected

bridge

test5.AprilTagPianoDetector.bridge

device

test5.AprilTagPianoDetector.device

image_callback

test5.AprilTagPianoDetector.image_callback

ordered_keys

test5.AprilTagPianoDetector.ordered_keys

publisher_keys

test5.AprilTagPianoDetector.publisher_keys

publisher_tag

test5.AprilTagPianoDetector.publisher_tag

subscription

test5.AprilTagPianoDetector.subscription

tag_detector

test5.AprilTagPianoDetector.tag_detector

yolo_model

test5.AprilTagPianoDetector.yolo_model

---

The documentation for this class was generated from the following file:

• src/perception/test5.py