How does OpenCV

OpenCV with Python

After updating to Pycharm 2019.1 there may be problems with ALtGr + [etc.

Workaround: In PyCharm click Ctrl + Shift + A. Then write registry and click on it. Find and uncheck it.

Raspberry Pi

OpenCV installation

  • Install a current version with everything from Raspbian on the SD card, set up everything, update the system via:
  • Install everything possible according to what OpenCV needs (everything has to be in one line):
sudo apt install libatlas3-base libsz2 libharfbuzz0b libtiff5 libjasper1 libilmbase12 libopenexr22 libilmbase12 libgstreamer1.0-0 libavcodec57 libavformat57 libavutil55 libswscale4 libqtgui4 libqt4-test libqtcore4 libwebp6
  • Then install OpenCV (and if it is missing numpy) via pip3, here as root, in the fattest flavor:
  • Test whether numpy and opencv are installed correctly. To do this, open on the console and try out whether the following two commands do not return an error:

    If error messages are displayed during import cv2, it helps to install the following libs:

    sudo apt-get install libcblas-dev sudo apt-get install libhdf5-dev sudo apt-get install libhdf5-serial-dev sudo apt-get install libatlas-base-dev sudo apt-get install libjasper-dev sudo apt-get install libqtgui4 sudo apt-get install libqt4-test
    • Tip: As an IDE under Raspbian for Python3 Thonny worth a try:
      • Installation best via pip:
      • After starting Thonny under "Tools-> Options ...-> General", set the UI mode to "expert". A dark theme can also be set at the same time, then it looks good.
      • The thing can look at variables, set breakpoints, display variable contents and install packages and shows suggestions for clean programming - thus a replacement for the PyCharm, which does not work under Raspberry.

    Camera installation

    • Activate the camera:, then activate the camera under "5 Interfacing Option". Quit tool via "Finish", restart Raspberry
    • Test the camera via: The video stream should be visible for 5 seconds.
    • The Raspberry camera should also be integrated as a V4L2 device so that standard examples can also run. To do this, do the following:
      • Edit the file / etc / modules as root via
      • Add a line with the content at the end of the file, then save the file with CTRL + o, exit the editor with CTRL + x, restart the system
      • Then the command should find a device file and not output an error. If not, it may help in the above step before the bcm2835-v4l2-Line insert a line with the following content: (also here after rebooting). For me, however, that wasn't necessary.
    • Now everything can be tested together by starting the script in the section OpenCV_mit_Python # Tests_der_Kamera_unter_Python-OpenCV on the console or e.g. in Thonny. It should display a window with the camera image in grayscale.

    Windows (and possibly also Mac / Linux) via PyCharm

    • Install Python3 from Important:
      • Install as admin with a right click
      • When installing, also have pip installed and paths set
    • Install PyCharm Community Edition from Jetbrains
      • I then created a new project in it, choosing "Virtualenv" as the environment
      • Open the Settings window via "File-> Settings ...",
        • Then open your own "Project: xxx" on the left and go to "Project Interpreter" below
        • A Python3 version should now be selected and at least "pip" should be available under the packages
        • Using the small "+" symbol on the far right, further packages can now be installed for the project, a window "Available Packages" opens
          • Enter "opencv" in the search and the package "opencv-contrib-python" install - below it will be shown somewhere if it worked
          • Then search for "numpy" and the package "numpy" to install
          • Then close the packet manager window
        • Also close the Settings window with "OK"
      • You probably have to wait a little longer because PyCharm is still installed in the background. You can only start when PyCharm is finished.

    The following script should run under all operating systems, under Linux / Raspbian a V4L2 device is required, under Windows it worked directly. Try the following script on the respective machine - via Terminal, Thonny, PyCharm or whatever:

    import numpy as np import cv2 as cv cap = cv.VideoCapture (0) while (True): # Capture frame-by-frame ret, frame = () # Our operations on the frame come here gray = cv.cvtColor (frame, cv.COLOR_BGR2GRAY) # Display the resulting frame cv.imshow ('Hit ESC to close', gray) if cv.waitKey (1) & 0xFF == 27: break # When everything done, release the capture cap.release () cv.destroyAllWindows ()

    The video image should be displayed in grayscale, press "ESC" in the active camera image window to exit the program properly.

    RAW access to the Raspberry Pi camera

    The following script can be used to access the Raspberry Pi camera without a V4L2 driver. Possibly this also enables finer setting options of the camera such as exposure time etc.

    # OpenCV camera test special for Raspberry Pi # import the necessary packages from picamera.array import PiRGBArray from picamera import PiCamera import time import cv2 import numpy as np # initialize the camera and grab a reference to the raw camera capture camera = PiCamera () camera .resolution = (640, 480) camera.framerate = 32 camera.rotation = 0 rawCapture = PiRGBArray (camera, size = camera.resolution) # allow the camera to warmup time.sleep (1) # capture frames from the camera for frame in camera.capture_continuous (rawCapture, format = "bgr", use_video_port = True): # grab the raw NumPy array representing the image, then initialize the timestamp # and occupied / unoccupied text image = frame.array # show the frame gray = cv2 .cvtColor (image, cv2.COLOR_BGR2GRAY) cv2.imshow ("Hit ESC to end", gray.astype (np.uint8)) # clear the stream in preparation for the next frame rawCapture.truncate (0) # if the `q `or ESC key was pressed, break from the loop key = cv2.waitKey (1) & 0xFF if key == ord ("q") or key == 27: break cv2.destroyAllWindows () camera.close ()

    The video image should be displayed in grayscale, press "ESC" in the active camera image window to exit the program properly.

    • Tutorials here:
    • Convert a colorful image to grayscale:
    • Convert grayscale image into color image: (it remains gray, but can then be colored)
    • To paint a box in a picture:
    • Paint text in the picture: (Larger font: cv2.FONT_HERSHEY_SIMPLEX)
    • Copy image area:
    • Insert image area somewhere else (size must fit exactly!):
    • To change all color components of a pixel:
    • Change only one color component (here g = index 1 because sequence BGR):
    • Get 8-bit color components: (if you have calculated with float or something)
    • ArUco marker (similar to QR code).
    • ArUco's can be created with opencv or online here

    Mouse events

    Mouse events can be processed via a callback, but OpenCV would like to know which of the "imshow" windows should be used for this. To do this, the window must be explicitly given a name at the start.

    mx = 10 my = 10 def mouseEvent (event, x, y, flags, param): global mx, my mx = x my = y #print (x, y) cv.namedWindow ("Window1") cv.setMouseCallback (" Window1 ", mouseEvent)

    Read bar / QR codes

    A working library for this is "zbar" (to be found with PIP as "pyzbar"). However, the library still requires "PIL" (Python Imaging Library), which is currently being continued as "PILLOW". In order to be able to execute the examples of "zbar", the wrapper "PILLOW - PIL" must be installed in addition to "PILLOW".

    decoded = pyzbar.decode (im) for obj in decoded: print ('Type:', obj.type) print ('Value:',, '\ n')

    Optical flow

    Example code to track features, aka optical flow. Code works on OpenCV version 3.4.4



    ArUco - Pose Estimation

    Finds the ArUco marker and its position and orientation in space.
    A distance function can then be used, for example, to determine the distance between the center of the marker and the camera. This is displayed in cm at the bottom left.

    import numpy as np import cv2 import cv2.aruco as aruco cap = cv2.VideoCapture (0) aruco_dict = aruco.Dictionary_get (aruco.DICT_6X6_250) parameters = aruco.DetectorParameters_create () while (True): ret, frame = ( ) gray = cv2.cvtColor (frame, cv2.COLOR_BGR2GRAY) corners, ids, rejectedImgPoints = aruco.detectMarkers (gray, aruco_dict, parameters = parameters) gray = aruco.drawDetectedMarkers (gray, corners, ids) mtx = np.array ([ [5.3434144579284975e + 02, 0., 3.3915527836173959e + 02], [0., 5.3468425881789324e + 02, 2.3384359492532246e + 02], [0., 0., 1.]], np.float) dist = np. array ([- 2.8832098285875657e-01, 5.4107968489116441e-02, 1.7350162244695508e-03, -2.6133389531953340e-04, 2.0411046472667685e-01], np.float) if ids! = None: # if aruco marker detected, rvec objp = aruco.estimatePoseSingleMarkers (corners, 3.5, mtx, dist) # For a single marker aruco.drawDet ectedMarkers (gray, corners, ids, (0, 255, 0)) aruco.drawAxis (gray, mtx, dist, rvec, tvec, 10) print (tvec [0] [0]) distance = round (np.linalg. norm (tvec [0] [0]), 2) cv2.putText (gray, str (distance) + "cm", (10, 400), cv2.FONT_HERSHEY_PLAIN, 1, (255, 0, 255)) cv2. imshow ('frame', gray) if cv2.waitKey (1) & 0xFF == ord ('q'): break # When everything done, release the capture cap.release () cv2.destroyAllWindows ()