How to go from Camera Pixels to Real-World Coordinates (calibration)

💡 How-to
, ,
1

From Camera Pixels to Real-World Coordinates in Robot Coordinate System

Step 1: Calculate the Camera Matrix and Conversion Factor

    1. Create a camera matrix:
      - This website has all the necessary info for this:

      OpenCV Camera Calibration

    1. Determine the Conversion Factor:

      • Measure the width of the camera image in millimetres.

      • Divide the width in millimetres by the number of pixels of the image to get the conversion factor, so you can convert pixel measurements to millimetres.

Step 2: Convert Pixel Coordinates to Millimetres

You can use the conversion factor to convert pixel coordinates to millimetres:

  • x_mm = pixel_x * conversion factor

  • y_mm = pixel_y * conversion factor

This step gives you the coordinates of, say, a bottle in millimetres.

Step 3: Determine the Offset between Camera and Robot coordinates

Since the camera and the robot have a different reference point (origin), calculate an offset:

  1. Identify the reference point (0,0) in the camera:

    • For example, this can be a fixed point on the working surface. Or if you have mounted the camera on the robot, this is the point where the object you see has the coordinates (0,0).

  2. Determine the coordinates of this point in the robot coordinate system:

    • Measure where this point is relative to the robot origin and call this distance the Offset.

You always measure this Offset from the same point. For example, you call this point StartPos (from start position).

Step 4: Check that the axes match

Make sure the x and y axes of the camera and the robot are the same:

  • Check if the axes are opposite or mirrored. Sometimes you need to reverse or mirror the coordinates depending on the configuration of the camera and robot.

Step 5: Convert Object coordinates to Robot coordinates

Now that you have the coordinates of a detected object (such as a bottle) in camera coordinates (in millimetres), you can convert them to robot coordinates. Follow these steps:

  1. Take the x and y coordinates of the object in millimetres (calculated in step 2).

  2. Add the appropriate offsets to these values.

Now you know the position of your detected object from your StartPos.

Step 6: Add the Current Robot Position

Now you need to include the current position of the robot. This is needed to determine the object’s final position in world space, taking into account the robot’s StartPos.

  1. Determine the current position of the robot in the x and y directions (in millimetres), relative to its original starting position.

    • You do this by subtracting the current robot coordinates from your StartPos coordinates.

  2. Add this current position (step 6.1) to the object’s coordinates (x and y) that you calculated in step 5.

Formula of pickup coordinates

  • The pickupX and Y are the robot coordinations
  • The deltaX and Y are the coordinates of the object in mm
  • The offsetX1 and Y1 are the offset calculated in step 3
  • The array1[0] and array1[1] is the current position of the robot
  • The firsposX1 and Y1 are the positions mentioned in step 3 (StartPos

Congratulations now you have determined the coordinates of the object in world space.

1 Like