Working with masked images (ignoring parts of the image) # 1.1.4

New in version 1.1.4.

This tutorial demonstrates how you can ignore parts of images, that might vary in different scenarios (background, inner parts of GUI elements, …) in the search process of SikuliX.

We call it Masking of Images.

It comes in 3 different flavors (see Pattern):

  • using images with 100% transparent parts (auto-masked)
  • saying, that the black parts of an image should be ignored (black-masked)
    Pattern(imageWithBlackParts).mask()
  • using an auto-masked or black-masked image as the mask (image-masked)
    Pattern(someImage).mask(imageWithBlackParts)
1# original image
2shot = 
3
4# image with transparency (auto-masked)
5shotTrans = 
6
7# image with black parts useable as mask (black-masked)
8shotBlack = Pattern().mask()
9
10# image with black/white parts useable as mask
11shotMask = 
12
13# Image masked with black/white image shotMask (image-masked)
14shotMasked = Pattern().mask(shotMask)
15
16# find the search area
17# the image shot-tile-small.png must be visible on screen
18reg = None
19for match in findAllList(shot):
20  if reg: reg = reg.union(match)
21  else: reg = match
22if reg:
23  reg = reg.grow(10)
24  reg.highlight(2, "green")
25else:
26  print "search area not visible"
27  exit(-1)
28
29# a generalized function for this demonstration
30def search(image, descr = ""):
31  print "##### searching for: (%s) %s" % (image, descr)
32  matches = reg.findAllList(image)
33  if len(matches) > 0:
34    scoreMax = 0
35    scoreMin = 1
36    for match in matches:
37      match.highlight()
38      score = match.getScore()
39      if score > scoreMax: scoreMax = score
40      if score < scoreMin: scoreMin = score
41        print "    found: %d matches score max/min: %.4f / %.4f" % (len(matches), scoreMax, scoreMin)
42    else:
43    print "    found: no matches"
44    wait(2)
45  highlightOff()
46
47# showcase original image
48search(shot, "original image")
49
50# showcase auto-masked
51search(shotTrans, "transparency (auto-masked)")
52
53# showcase black-masked
54search(shotBlack, "black parts as mask (black-masked")
55
56# showcase image-masked
57search(shotMasked, "masked with other image (image-masked)")

This image must be visible on the screen, when running the example.

../../_images/shot-tile-small.png

… and this is the output you should get principally:

##### searching for: (shot.png) original image
    found: 6 matches score max/min: 0.9971 / 0.9971
##### searching for: (shotTransparent.png) transparency (auto-masked)
    found: 6 matches score max/min: 0.9907 / 0.9900
##### searching for: (P(shotBlack.png) S: 0.7 masked) black parts as mask (black-masked
    found: 6 matches score max/min: 0.9985 / 0.9983
##### searching for: (P(shot.png) S: 0.7 masked) masked with other image (image-masked)
    found: 6 matches score max/min: 0.9985 / 0.9983