blogging pywinauto

I have been quiet for a while - and no new releases of pywinauto for a bit either.

I finally figured out that my metaclass problem was not a metaclass problem at all - but a logic problem in my code :-)

I merged in that code into the trunk - and I should do another release soon (a small one!). This week has been busy - been in training almost all week - so that is feeding my programming need (and making me not want to touch the computer - some direct user queries always get me going - so if you have one let me know).

The next few weeks will be very busy

• with maybe a trip to San Francisco
• my parents coming on the 19th and being around for a week
• the day they leave then off on vacation :-) (for 10 days)

So next release will be the last one for some week (probably).

Finally my reason for posting - Jeff has done a nice screencast of pywinauto at:
http://showmedo.com/videos/video?name=UsingpyWinAutoToControlAWindowsApplication&fromSeriesID=7

Thank you Jeff!

I just released another version of pywinauto - and I am quite happy with it. I even squeezed some more performance improvements (it is now approximately 10-20% faster then the previous release - though I think it will be waiting for the application itself to respond most of the time!).

One thing that I had wanted to make into this release was a change from using a function to return the correct control Wrapper class, to having a __metaclass__ attribute in the base class, and add functionality to it's __new__ method.

something like: (paraphrased - not run!)

class MetaWrapper(type):
   registry = {}

   # this is called when each class is defined
   def __init__(cls, name, bases, attrs):
       # add information defined in cls to the registry
       # so that later we can return the correct class

   def GetWrapper(handle):
       # look in registry for correct class
       return correct_class
   GetWrapper = staticmethod(GetWrapper)

class HwndWrapper(object):
   __metaclass__ = MetaWrapper

   # called before each instance is created
   def __new__(cls, handle):
       correct_class = cls.GetWrapper(handle)

       # need to call base __new__ ?
       obj = correct_class.__new__(correct_class)

       # as we are returning a different class from HwndWrapper we need to
       # initialize it
       obj.__init__(handle)
       return obj

   def __init__(self, handle):
       # rest of stuff here

But I was getting errors I could not explain! Now that I write that out - there were times when I was returning a HwndWrapper instance, would that mean that it was getting initialized twice? I don't think that would have caused the problems I was seeing (but then again - I can't think of anything else either!)

Maybe one of you out there who knows this magic better can set me on the right track. Is this even something that I should be using - or are meta classes too much magic most of the time?

Anyway - happy pythoneering :-)

Well I finally managed it. (well 80% of it!).

What am I talking about? 'Application data' where you save information from one run of an automated GUI run and use that information next time to work with changed software (in my case Translated to a different spoken language!).

There is still one small part that I want to implement before I release a 0.3.0 release. Currently it works for dialogs, controls and menu's - but I still need to implement looking up appdata for selecting items in controls (listboxes, treeviews, etc)

The implementation is very basic - and certain things will not work without some form of user override (e.g. selecting the correct item in a sorted listbox/combobox). Unfortunately there is no override available (but I think I will need to look into that soon).

The internal handling of appdata may well change (almost certainly) - but there is very little that a user uses at the moment - so not a big impact I think!

This code does not affect normal operation - so I am not worried about this code causing regressions in the normal use case if just automating a GUI.

So for now I feel I have a working proof of concept - but I think I need to put more work into making it a robust, workable solution.

Wow - seems like I have at least one passionate user.

Jeff has been helping my make my wiki suck less :-) and he put together a nice little screencast of using pywinauto to automate Notepad.

Please go watch it at: http://pywinauto.pbwiki.com.

He has been coaching me a bit on marketing too - though I can't say pywinauto has the same problem as Python ;-). Though I feel that pywinauto is on the same level as many commercial automation tools, and it uses Python which is so far beyond many automation scripting languages.

Have a good weekend everyone.

Ever found it annoying when you start an application that takes a long time to start up , start working on other things and the application decides to popup and annoy you after/while it starts up?

Well you can now do the same with pywinauto :-)

I had tried to get this right before and hadn't succeeded - but I managed it today. HwndWrapper.SetFocus will now make the window the foreground window - even if it is not the current foreground window.

This was easy enough really - and I had thought that I had tried this already - but obviously hadn't :-(

Here is the new SetFocus method...

    def SetFocus(self):
      """Set the focus to this control

      Activate the window if necessary"""

      # find the current foreground window
      cur_foreground = win32functions.GetForegroundWindow()

      # if it is already foreground then just return
      if self.handle != cur_foreground:

          # get the thread of the window that is in the foreground
          cur_fore_thread = win32functions.GetWindowThreadProcessId(
              cur_foreground, 0)

          # get the thread of the window that we want to be in the foreground
          control_thread = win32functions.GetWindowThreadProcessId(self, 0)

          # if a different thread owns the active window
          if cur_fore_thread != control_thread:
              # Attach the two threads and set the foreground window
              win32functions.AttachThreadInput(
                  cur_fore_thread, control_thread, True)

              win32functions.SetForegroundWindow(self)

              # detach the thread again
              win32functions.AttachThreadInput(
                  cur_fore_thread, control_thread, False)

          else:   # same threads - just set the foreground window
              win32functions.SetForegroundWindow(self)

          # make sure that we are idle before returning
          win32functions.WaitGuiThreadIdle(self)

          # only sleep if we had to change something!
          time.sleep(.06)

      return self

And here is a short bit of code you can test it with

from pywinauto.application import Application
import time

app = Application.start("Notepad.exe")
app.UntitledNotepad.MenuSelect("Format->Font")

for i in range(4):
  time.sleep(2)
  app.Font.Edit1.SetFocus()

If you run that code and work with other windows - then the Notepad Font dialog will popup (annoyingly :-) ) every 2 seconds.

All is left now is to add a call to this method in those other methods that REQUIRE the window to have focus (TypeKeys() for sure, maybe the ...Input() methods - but they probably require a WindowFromPoint/RealChildWindowFromPoint/ChildWindowFromPoint call first to ensure it's necessary).

(all happy with myself for getting that off the todo list - except it wasn't on the todo list :-D )

I wanted to roll up my various changes before I give my presentation to Boston-Pig meeting this Thursday.

Also I will be traveling next Monday to Europe for work, that might mean that I can do more work on pywinauto - but it also might mean that I can do much less.

Going forward I have more refactoring to do (probably code breaking - but I can leave methods in and deprecate them for a release or two), and I still need to work on the functionality that allows a script to work unchanged on different langauges.

Have fun :-) - and don't worry about asking me questions or letting me know if you are using pywinauto - I really really don't get a lot of mails :-) (It is only since releasing my own package that I notice how little feedback I have given to others in the past!).

I just read an interesting list of "new laws" at http://pfeifferreport.com/trends/trend_userexperience.html (found from http://pyre.third-bit.com/blog/archives/000414.html (found from http://planet.python.org/index.html) .

I find that a number of these resonate strongly with me, and put in words some of what I am trying to achieve with pywinauto.

I should print the list and post it above my desk - then when I go to refactor or add new code I can ask myself if the changes obey the rules :-) (I think 6, 8 and 9 are the ones that will be most useful day to day).

Getting the text of a container control is maybe not that critical for the everyday user of pywinauto - but it should be easy to find if they do need it. Here are some examples of how inconsistent this is at the moment...

  listbox.ItemTexts()[item_index]
  ListView.GetItem(item_index, subitem_index)['text']  # returned item is just a dict
  TreeView.GetItem(0,2,4).Text()
  Header.GetColumnText(col_index)
  Statusbar.GetPartText(part_index) # at least these last two are similar
  Rebar.GetBand(band_index).text # attribute set in returned ctypes structure!

These kinds of inconsistencies do not help anyone learn pywinauto (I hadn't realized it was so bad :-) )

Just as a counter point to above - it's not all so bad.
control.Select() is defined for most classes and behaves (hopefully) more or less as expected.

So - going forward I am going to try and ensure usability and consistency of pywinauto.

The other point is documentation - there IS documentation for pywinauto - but I definitely do not consider myself a tech writer. From reading the rules - it also occurred to me that my documentation is lacking in a major way - it is not easy to find what you are looking for.
I think a new set of pages which describe in a succinct way how to work with controls will help.

E.g.
Operations on controls:

ListBox
     Select
     SetFocus
     GetFocus
ListView
     Select
     IsSelected
     IsChecked
     Check
     UnCheck
     IsFocused
TreeView
     Select
     ...

Even the list by itself helps - in that it would allow you to see what is available more easily then browsing the current documentation.

I guess this little blog will get lost in the flood of PyCon blog stuff - or else everyone will be so busy at PyCon that this entry will stand out (whether that's a good thing or not I am not sure :-)

It sounds like there are one or two people actively using pywinauto - and I am getting some feedback, some bug reports, some suggestions - It's great - but I need to be organized so I don't loose it. So I am going to put some of it here...

First of all there is a Python meetup in Boston on Thursday 9th of March, and if I can get my act together (a Powerpoint presentation and a good demo I guess?) then I will be presenting pywinauto.

So here are the things I want to track

• Some applications implement menu's as Toolbars (Internet Explorer for one) and of course MenuSelect() or MenuClick() doesn't work on those. I need to research if there is some style or setting that says a toolbar button will actually popup a menu. If there is I can update the MenuSelect/MenuClick function to use them, but if not they you will have to document to use one of the following methods:
  app.dlg.TypeKeys("%FWF") # &File->Ne&w->&Folder
  or
  # this functionality is not implemented yet - still
  # need to change the interface to Toolbar buttons.
  app.dlg.Toolbar.Button("File").Click()
  app.PopupMenu.MenuSelect("New->Folder")
  
  For now I should just update the documentation (as I feel that will be the final outcome anyway). Especially as the example given was a control with class WTL_CommandBar (which is a different kettle of fish completely!)
  
• I wanted to add my reasoning for creating pywinauto to the documentation and say why I think it is better then a lot of automation tools out there (the idea at least - I hope the implementation doesn't let it down!)
• Some way of specifying unique text when looking for a control. For example if you look for a control with the text "Click this button to save as PNG" (maybe there are other buttons "Click this button to save as XXX", etc), then it would be nice if app.dlg.PNG.Click() would work - but it doesn't :-(. The reason is that the matching algorithm I use (using difflib) takes the overall text match - and the match of PNG in the overall text is low.
  The reason that I do not just select the best match (original code did do this) is that it matches sometimes when you don't want to. For example if you want to check if a dialog exists for example you could write
  
  if app.dlg.Exists():
  # do something because this dialog exists
  
  but if the searching is not strict enough it will find other dialogs and the script will be broken (and it might be hard to fix). Currently the way around is either specify more of the text:
  app.dlg.ClickbuttontosaveasPNG.Click()
  or use the window_ method...
  app.dlg.window_(title_re =".*PNG$").Click()
  
  One idea was possibly to allow something like
  app.dlg.PNG_IN_.Click()
  or maybe
  app.dlg[".*PNG$_RE_"].Click()
  where the _IN_ and _RE_ specify that the text should be 'in' or should be tested using a regular expression. I am not sure about either of these as they appear to be a bit 'magic'.
• Finish off the Wait* functions. Currently in application.WindowSpecification (which is the class you get if you do app.dlg, or app.dlg.ctrl) there are the following methods:
  Exists(self, timeout = exists_timeout))
  WaitReady(self, timeout = window_find_timeout, wait_interval = window_retry_interval)
  WaitNotEnabled(self, timeout = window_find_timeout, wait_interval = window_retry_interval)
  WaitNotVisible(self, timeout = window_find_timeout, wait_interval = window_retry_interval)
  
  These should all have the same signature, and then I need to fill out the rest of the methods ^Wait(Not)?(Enabled|Visible|Ready)$. Though just writing that and thinking of duplicated lines of code - maybe I should just have a Wait() method e.g.
  app.dlg.control.Wait(enabled = True, Visible = False)
  
  would return the control being waited for once it is both Enabled and Hidden (unlikely to be very useful :-) or return None on timeout (or should it raise an exception?)
  
• Clear up where to look for appropriate methods. Most methods are in the control wrapper for that particular control (or it's base class(es)) but some (important) methods are in WindowManager. So to be clear the available methods for a particular window are the union of
  - methods of WindowSpecification
  - methods of the Wrapper for this control
  - methods of the base class(s) of the wrapper
• Look into pyAA to help getting information from 'difficult' controls (and possibly other uses for it too!)
  
• Then all my various ToDo items:
  - Implement 'application data' that will allow the same script to run on many languages
  - Implement basic Recording functionality
  - Lots more of everything (wrapped controls, tests, examples, documentation, etc)