Develop Control Application
The Challenge
The task that you are working on will be to enable your robot to identify a fedora on the floor. Your robot should drive towards the fedora and stop shortly before it.
The basic flow for your app will be:
-
retrieving an image
-
sending it to the object detection service
-
analyzing the results
-
detected class
-
confidence score
-
coordinates of the bounding boxes in your camera image
-
-
working with the result to determine where the robot should move to next
MVP (Minimal Viable Product)
To speed things up we will give you a solution as an MVP.
Change your code to this :
# Drop your code here
turn_counter = 0
while thread_event.is_set():
objects = take_picture_and_detect_objects()
coordinates = find_highest_score(objects)
if coordinates and coordinates.confidence_score > 0.5:
print(f'''Object with highest score -> [
confidence score: {coordinates.confidence_score},
x upper left corner: {coordinates.x_upper_left},
y upper left corner: {coordinates.y_upper_left},
x lower right corner: {coordinates.x_lower_right},
y lower right corner: {coordinates.y_lower_right},
object class: {coordinates.object_class} ]''')
move_x = (coordinates.x_upper_left + coordinates.x_lower_right) / 2
print(f'move_x: {move_x}')
if move_x < 320:
turn_left(10)
else:
turn_right(10)
delta = coordinates.x_lower_right - coordinates.x_upper_left
print(f'delta: {delta}')
if delta < 350:
move_forward(10)
else:
print('Done - arrived at the object')
return
else:
if turn_counter < 360:
turn_left(10)
turn_counter = turn_counter + 10
else:
print('Done - no object found')
return
print('Done')What does this code do, you ask?
Well first of all it keeps on looping until someone presses the Stop button. It takes a picture from the camera, finds the Fedora object with higest confidence score.
Then it looks for the center of the bounding box. Depending on the position of the center point in the camera image, the code decides if the robot needs to turn left or right to align it with the fedora.
Then it just keeps on driving until the bounding box has a certain size, which means that you are close enough to the Fedora.
Now place a Fedora somewhere near your robot and see if it can find it.
Dev & AI Collaboration
Have look at this code. Perhaps there is something you can optimize?
Think about these points :
-
How can I align the robot towards the fedora with the coordinate information
-
How do I know when to stop the robot
-
How can I optimize the "think" and "react" phases
-
How can I handle corner cases
-
Perhaps the model should be tuned increase detection results?
|
Do not go beyond this point until your code is fully prepared for Challenge Level 1 
|
Pushing your Code
Currently your code only resides in your Workspace, which is called the "Inner Loop" in Development terms. To publish and build you will need to push your code to your Git repository.
To be able to push you need to set up your git email
-
Open a new Terminal by click on the + Icon in the bottom right and choosing New Terminal
-
enter
git config --global user.email "pilot@robot.to" git config --global user.name "pilot"
To push your code:
-
In the Dev Spaces menu on the left click on the Git icon
-
Your modified files will be shown
-
Select the relevant files (
app.pyandconfig.py)
-
Right Click and select Stage Changes
-
Enter a comment in the top field
-
At the blue Commit button, click on the downwards bracket at the very right and choose Commit & Push
-
On the top a prompt dialog will open to enter your Gitea username and password
-
username:
team-1 -
password :
secret-passwordNow you code has been pushed to your repo. You can check in Gitea if you want.
-