|
| 1 | +# Python grab screen tutorial |
| 2 | +### Original text version of tutorial you can visit [here](http://pylessons/Tensorflow-object-detection-grab-screen/). |
| 3 | + |
| 4 | +[](https://www.youtube.com/watch?v=Wc2gFarBEE0 "TensorFlow object detection tutorial how to grab a screen") |
| 5 | + |
| 6 | +In previous installation tutorial we mentioned that vision of this tutorial series is to create TensorFlow object detection model, that could detect CS:GO players. In this tutorial we’ll focus more on how to grab our monitor screen where we could detect objects. I must mention that we need to find the fastest way to grab screen, because later when we process images FPS drops, and if our screen grab FPS would be slow, this would affect our final frames per second. |
| 7 | + |
| 8 | +At first we need to install all required libraries, so you can begin from installing opencv by writing this line: |
| 9 | +``` |
| 10 | +pip install opencv-python |
| 11 | +``` |
| 12 | +then you can install mss library: |
| 13 | +``` |
| 14 | +python -m pip install --upgrade --user mss |
| 15 | +``` |
| 16 | +If you don't have already, install numpy: |
| 17 | +``` |
| 18 | +pip install numpy |
| 19 | +``` |
| 20 | +And at the end probably you will need pywin32 package, [download](https://github.com/Sentdex/pygta5/blob/master/grabscreen.py) it and install from wheel file. grabscreen.py file you can download from this GitHub repository. |
| 21 | +Now you should be ready to test grabscreen codes. So begin your code by importing libraries and setting variables that we'll use: |
| 22 | + |
| 23 | +``` |
| 24 | +import time |
| 25 | +import cv2 |
| 26 | +import mss |
| 27 | +import numpy |
| 28 | +from PIL import ImageGrab |
| 29 | +from grabscreen import grab_screen |
| 30 | +
|
| 31 | +# title of our window |
| 32 | +title = "FPS benchmark" |
| 33 | +# set start time to current time |
| 34 | +start_time = time.time() |
| 35 | +# displays the frame rate every 2 second |
| 36 | +display_time = 2 |
| 37 | +# Set primarry FPS to 0 |
| 38 | +fps = 0 |
| 39 | +# Load mss library as sct |
| 40 | +sct = mss.mss() |
| 41 | +# Set monitor size to capture to MSS |
| 42 | +monitor = {"top": 40, "left": 0, "width": 800, "height": 640} |
| 43 | +# Set monitor size to capture |
| 44 | +mon = (0, 40, 800, 640) |
| 45 | +``` |
| 46 | +We will begin with most basic and slowest PIL method. In this first code part I commented all lines, I wrote what it is done in each line and in other examples I simply copied PIL code and changed few lines of code, exactly that you can see on my youtube tutorial. |
| 47 | +``` |
| 48 | +def screen_recordPIL(): |
| 49 | + # set variables as global, that we could change them |
| 50 | + global fps, start_time |
| 51 | + # begin our loop |
| 52 | + while True: |
| 53 | + # Get raw pixels from the screen, save it to a Numpy array |
| 54 | + img = numpy.asarray(ImageGrab.grab(bbox=mon)) |
| 55 | + # Display the picture |
| 56 | + cv2.imshow(title, cv2.cvtColor(img, cv2.COLOR_BGR2RGB)) |
| 57 | + # add one to fps |
| 58 | + fps+=1 |
| 59 | + # calculate time difference |
| 60 | + TIME = time.time() - start_time |
| 61 | + # check if our 2 seconds passed |
| 62 | + if (TIME) >= display_time : |
| 63 | + print("FPS: ", fps / (TIME)) |
| 64 | + # set fps again to zero |
| 65 | + fps = 0 |
| 66 | + # set start time to current time again |
| 67 | + start_time = time.time() |
| 68 | + # Press "q" to quit |
| 69 | + if cv2.waitKey(25) & 0xFF == ord("q"): |
| 70 | + cv2.destroyAllWindows() |
| 71 | + break |
| 72 | +``` |
| 73 | +Here I used sentdex method of taking computer screen, it's much faster that PIL. But we must have grabscreen.py file in our local files to use it, so we move to final example. |
| 74 | +``` |
| 75 | +def screen_grab(): |
| 76 | + global fps, start_time |
| 77 | + while True: |
| 78 | + # Get raw pixels from the screen |
| 79 | + img = grab_screen(region=mon) |
| 80 | + # Display the picture |
| 81 | + cv2.imshow(title, cv2.cvtColor(img, cv2.COLOR_BGR2RGB)) |
| 82 | + fps+=1 |
| 83 | + TIME = time.time() - start_time |
| 84 | + if (TIME) >= display_time : |
| 85 | + print("FPS: ", fps / (TIME)) |
| 86 | + fps = 0 |
| 87 | + start_time = time.time() |
| 88 | + # Press "q" to quit |
| 89 | + if cv2.waitKey(25) & 0xFF == ord("q"): |
| 90 | + cv2.destroyAllWindows() |
| 91 | + break |
| 92 | +``` |
| 93 | +This is the last example of taking computer screen, and I love it the most. Because for this method we don't need local files and most importantly, this method has more functionality. In their website you can find that it's possible to use this method to grab screen from different computer screens, must mention that this is impossible with previous methods. Moreover this method is as fast as second example. |
| 94 | +``` |
| 95 | +def screen_recordMSS(): |
| 96 | + global fps, start_time |
| 97 | + while True: |
| 98 | + # Get raw pixels from the screen, save it to a Numpy array |
| 99 | + img = numpy.array(sct.grab(monitor)) |
| 100 | + # to get real color we do this: |
| 101 | + img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) |
| 102 | + cv2.imshow(title, cv2.cvtColor(img, cv2.COLOR_BGR2RGB)) |
| 103 | + fps+=1 |
| 104 | + TIME = time.time() - start_time |
| 105 | + if (TIME) >= display_time : |
| 106 | + print("FPS: ", fps / (TIME)) |
| 107 | + fps = 0 |
| 108 | + start_time = time.time() |
| 109 | + # Press "q" to quit |
| 110 | + if cv2.waitKey(25) & 0xFF == ord("q"): |
| 111 | + cv2.destroyAllWindows() |
| 112 | + break |
| 113 | +``` |
| 114 | +Here just simply uncomment function line, which you would like to test. |
| 115 | +``` |
| 116 | +screen_recordMSS() |
| 117 | +#screen_recordPIL() |
| 118 | +#screen_grab() |
| 119 | +``` |
| 120 | +In this short tutorial we learned 3 different ways to grab computer screen. It's sad that maximum performance we can get is around 20 FPS, but this is the best I found right now. If someone know better ways how to get more FPS, please let me know. So now we can move to another TensorFlow tutorials. |
| 121 | +### Original text version of tutorial you can visit [here](http://pylessons/Tensorflow-object-detection-grab-screen/). |
0 commit comments