mark.py

Creating images of python code + pygments = WIN!

2011-06-07T17:46:00.003+08:00

Posted a short script to http://paste.pocoo.org/show/402065/

Output PNG files using Pygments for the python code in a directory (used it for short snippets used in a presentation) :)

(edit: Thanks to Marius for suggesting a sample.)

Sample has one function highlighted (any line you want highlighted put ## at the end of the line).

Currently the script has very few options - just what I needed RIGHT NOW :)

Compiling Python modules on windows manually

2010-07-13T15:06:00.002+08:00

So I wanted to use Pystemmer (http://snowball.tartarus.org/wrappers/PyStemmer-1.1.0.tar.gz not http://sourceforge.net/projects/pystemmer/ which seems to be out of date), but there is no included Visual Studio makefile or solution.

So what to do?

Below are the steps that I find work reasonably well (hopefully documented in a way that will allow other people to do the same for whatever library they are working with).

Note - I assume the following:

The code as provided will compile with Visual C (if Visual C cannot compile it - then your job is way harder!)
No extra tools (autoconf, etc) are required

So the steps are:

Ensure directory with Python.h is in the %include% directories (menu item: Tools -> Options, Projects and Solutions -> VC++ Directories, Show directories for -> Include Files)
Ensure directory with Python.lib is in the %lib% directories (as as steps for Include files - just select Library Files instead at the end)
In Visual Studio 2008 (for Python 2.6) select menu item File -> New -> Project From Existing Code...
Select the directory with the downloaded code as the Project File Location
Give the project a name (generally a good idea to give it the name of the Python module - i.e. in my example "Stemmer" )
Ensure that the Release build is selected (if using the toolbar) because generally on windows you will not have the debug library: python_d.lib.

Modify the project properties:
Set General -> Configuration Type to "Dynamic Library (.dll)"
Set file extenstion for Linker -> General -> Output File to ".pyd"

Try and compile - here are some of the common problems and possible solutions:
- "multiply defined symbols"
  
  libstemmer_utf8.obj : error LNK2005: _sb_stemmer_length already defined in libstemmer.obj
  ...
  Release\Stemmer.pyd : fatal error LNK1169: one or more multiply defined symbols found
  
  The reason for this is that a symbol is defined in 2 source files, for PyStemmer this was because there is both libstemmer_utf8.c and libstemmer.c in the codebase which define the same symbols. The fix was to remove one from the project
- "Cannot open include file"
  
  .\src\Stemmer.c(30) : fatal error C1083: Cannot open include file: 'libstemmer.h': No such file or directory
  
  This is a simple fix - find where the include file is and add it to the include path Project Properties dialog, C/C++ -> General -> Additional Include Directories

This is as much a reminder for me - but if this helps you great :)

SendKeysCtypes release (for some value of released)

2010-04-25T13:29:00.003+08:00

I finally got around to releasing SendKeysCtypes (Sending key strokes to the active app on Windows) to http://bitbucket.org/markm/sendkeysctypes.

There is no setup.py yet, it is not registered on PYPI - but at least it is available.

I did a quick check and had to make a small change to make it work on Python 3.1 :)

Give it a try.

I would like to thank Oliver Rutherford for his sendkeys module (http://www.rutherfurd.net/python/sendkeys/) which I have used for years as part of pywinauto.

I only just realized while writing this that it is also posted to bitbucket http://bitbucket.org/orutherfurd/sendkeys)

The main benefits between this module and Oliver's that I can think of are:

No compilation required (works on Python 3)
Support for Unicode Characters

Code that works for one of these modules should work for the other also


import SendKeysCtypes
import os, time
os.system("start  notepad.exe")
time.sleep(.5)
SendKeysCtypes.SendKeys("Hello in  Notepad", with_spaces = True)
SendKeysCtypes.SendKeys("%f")  # ALT + F  "File" menu
SendKeysCtypes.SendKeys("x")   # Exit

I am speaking at PyCon Asia Pacific in June

2010-04-18T11:52:00.003+08:00

I will be speaking at PyCon Asia Pacific.

My talk is titled "Writing Application Plugins in Python". I aim to show how Python can be used to implement plugin functionality for applications that never even heard of Python.

The idea being, that if you write a small amount of shim code (using Pyrex/Cython to make it really easy) then you can pass most of the work off to Python. I work on Windows, and my examples will be on that OS, but very little of what I present will be specific to Windows.

I am still searching around for a good Open Source application with a reasonably simple API/SDK that I can demonstrate with. Leave a comment if you have a suggestion. But the majority of the talk will be generic to any application giving introductions to the various tools and modules used.

If you are in or around Singapoare in June - or feel like seeing South East Asia - then drop by :)

A small library wants to break out of pywinauto

2010-04-18T11:41:00.003+08:00

I have been working on pywinauto for about 4 and a half years now and its not really moving in any direction. I make small tweaks here and there and make releases from time to time - but nothing major.

This was the first open source project that I released and I think maybe I was a little over-enthusiastic :). I had hoped that I would be using it myself, I also wished it to be functional and easy enough thatthat it could replace expensive commercial software with similar functionality (but with their own annoying languages). But I am no longer working with said software and my main focus went to other things.

I believe now that pywinauto should be split. There is at least 1 small library (and probobaly more) that should be freed from the overall project.

Get information from windows controls (Maybe of the questions on the pywinauto list is how to get the values from text boxes, etc.)
A library for automating Windows, not an auotmation framework/language - a library. Meaning that it is up to the user to delay between clicks for windows to open/close. It is up to the user how they want to select controls.
SendKeysCtypes (I re-wrote Python SendKeys so that it would support Unicode and not need to be compiled (uses Ctypes). This is more or less ready to release - I just haven't gotten around to doing that :(. (if you need it - grab it from Pywinauto SVN)
pywinauto - which would use these previous components but would be much smaller itself, it would work as it does now, trying to help the user as much as possible in not having to worry about timing issues.

Seeing as it turned out that *I* don't really need an automation library/toolkit - it would be great to get some help with some of this work.

Other directions I am thinking of are providing modules so that pywinauto (or the gui automation libarary) can work as an almost direct replacement of things like AutoHotkey or AutoIT or use a similar API as LDTP.

I also need to fix the Pywinauto web presence - which is amazingly scattered - but this post is already long enough.

Added functions to betterbatch and debugging on the horizon

2010-03-07T22:20:00.002+08:00

I was expecting function to be not so hard (just another way of
running a few steps) but it took a little bit of time all the same.

I got it implemented (though no tests for it yet) and it seems OK, I
will know better how it works when I get around to using it (hopefully
in the next week). Weekends for the tool and weekdays for using the
tool in production :)

With this addition I think the language definition is more or less
complete I think. I can imagine tweaking it a bit, fixing bugs etc,
but I don't expect to implement many other language constructs.

One of the next things that I am planning to implement is a simple
debugger (step through statements, print variable value, maybe change
variable values, etc). I was expecting this to be a fairly major
undertaking but again Python and it's community come to the rescue.
Right on cue

a) Catherine Devlin mentioned cmd2 on her blog,
http://catherinedevlin.blogspot.com/2008/01/introducing-cmd2.html
which made me look at cmd in the Python standard library
b) I found the Python documentation a bit dry (or else I had very
little patience) so googled for some examples - and Doug Hellmann's
excellent PyMOTW (http://blog.doughellmann.com/2008/05/pymotw-cmd.html
if you don't know it - bookmark it!) popped up with some good examples
and great documentation.

I found that I was able to throw together a simple debugger (next,
print_var, print_vars, etc)

So this means that a debugger is really easy to write - and is a much smaller task than I was hoping. Maybe next weekend I get the debugger implementation finished.

BetterBatch released

2010-03-02T21:54:00.002+08:00

I have released BetterBatch (http://code.google.com/p/betterbatch/)
and I would like more people to know about it.

I created it as something that I could use at work. We have many tasks
that require adhoc automations which may have different people running
them. These automations may not have a very long life so it often
doesn't make sense to write it up as Python/Perl script. This work is
done on Windows machines so not many of us (me included) do not have a
lot of experience with shell scripting. Even if we did - shell
scripting wouldn't necessarily be the ideal candidate.

So it was designed as a batch file replacement with some improvements:
Some requirements I had while making this:

output from commands can be assigned to variables
it needs to be just as easy to call external applications as a batch file
all actions should be logged
the return value of everything should be checked (and stop the script on error!)
it should be prettier than a batch file

I have not tested this on any *nix style machine yet - but believe it
should work (meaning that it probably won't work at first - but I am
hoping that minimal changes will make it work)

Give it a try (please :) )
Download: http://code.google.com/p/betterbatch/downloads/list
Log an issue: http://code.google.com/p/betterbatch/issues/entry
Discuss: http://groups.google.com/group/betterbatch-discuss/topics

Screencast on ShowMeDo

2006-05-12T03:04:00.000+08:00

I have been quiet for a while - and no new releases of pywinauto for a bit either.

I finally figured out that my metaclass problem was not a metaclass problem at all - but a logic problem in my code :-)

I merged in that code into the trunk - and I should do another release soon (a small one!). This week has been busy - been in training almost all week - so that is feeding my programming need (and making me not want to touch the computer - some direct user queries always get me going - so if you have one let me know).

The next few weeks will be very busy

with maybe a trip to San Francisco
my parents coming on the 19th and being around for a week
the day they leave then off on vacation :-) (for 10 days)

So next release will be the last one for some week (probably).

Finally my reason for posting - Jeff has done a nice screencast of pywinauto at:
http://showmedo.com/videos/video?name=UsingpyWinAutoToControlAWindowsApplication&fromSeriesID=7

Thank you Jeff!

How to work with Metaclasses

2006-04-26T05:42:00.000+08:00

I just released another version of pywinauto - and I am quite happy with it. I even squeezed some more performance improvements (it is now approximately 10-20% faster then the previous release - though I think it will be waiting for the application itself to respond most of the time!).

One thing that I had wanted to make into this release was a change from using a function to return the correct control Wrapper class, to having a __metaclass__ attribute in the base class, and add functionality to it's __new__ method.

something like: (paraphrased - not run!)


class MetaWrapper(type):
   registry = {}

   # this is called when each class is defined
   def __init__(cls, name, bases, attrs):
       # add information defined in cls to the registry
       # so that later we can return the correct class

   def GetWrapper(handle):
       # look in registry for correct class
       return correct_class
   GetWrapper = staticmethod(GetWrapper)

class HwndWrapper(object):
   __metaclass__ = MetaWrapper

   # called before each instance is created
   def __new__(cls, handle):
       correct_class = cls.GetWrapper(handle)

       # need to call base __new__ ?
       obj = correct_class.__new__(correct_class)

       # as we are returning a different class from HwndWrapper we need to
       # initialize it
       obj.__init__(handle)
       return obj

   def __init__(self, handle):
       # rest of stuff here

But I was getting errors I could not explain! Now that I write that out - there were times when I was returning a HwndWrapper instance, would that mean that it was getting initialized twice? I don't think that would have caused the problems I was seeing (but then again - I can't think of anything else either!)

Maybe one of you out there who knows this magic better can set me on the right track. Is this even something that I should be using - or are meta classes too much magic most of the time?

Anyway - happy pythoneering :-)

Somethings are hard and something weren't mean to be easy - but nothing is impossible ;-)

2006-03-18T02:12:00.000+08:00

Well I finally managed it. (well 80% of it!).

What am I talking about? 'Application data' where you save information from one run of an automated GUI run and use that information next time to work with changed software (in my case Translated to a different spoken language!).

There is still one small part that I want to implement before I release a 0.3.0 release. Currently it works for dialogs, controls and menu's - but I still need to implement looking up appdata for selecting items in controls (listboxes, treeviews, etc)

The implementation is very basic - and certain things will not work without some form of user override (e.g. selecting the correct item in a sorted listbox/combobox). Unfortunately there is no override available (but I think I will need to look into that soon).

The internal handling of appdata may well change (almost certainly) - but there is very little that a user uses at the moment - so not a big impact I think!

This code does not affect normal operation - so I am not worried about this code causing regressions in the normal use case if just automating a GUI.

So for now I feel I have a working proof of concept - but I think I need to put more work into making it a robust, workable solution.

Passionate users, marketing and screencasts,

2006-03-11T04:01:00.000+08:00

Wow - seems like I have at least one passionate user.

Jeff has been helping my make my wiki suck less :-) and he put together a nice little screencast of using pywinauto to automate Notepad.

Please go watch it at: http://pywinauto.pbwiki.com.

He has been coaching me a bit on marketing too - though I can't say pywinauto has the same problem as Python ;-). Though I feel that pywinauto is on the same level as many commercial automation tools, and it uses Python which is so far beyond many automation scripting languages.

Have a good weekend everyone.

Setting the foreground window

2006-03-09T01:34:00.000+08:00

Ever found it annoying when you start an application that takes a long time to start up , start working on other things and the application decides to popup and annoy you after/while it starts up?

Well you can now do the same with pywinauto :-)

I had tried to get this right before and hadn't succeeded - but I managed it today. HwndWrapper.SetFocus will now make the window the foreground window - even if it is not the current foreground window.

This was easy enough really - and I had thought that I had tried this already - but obviously hadn't :-(

Here is the new SetFocus method...

    def SetFocus(self):
      """Set the focus to this control

      Activate the window if necessary"""

      # find the current foreground window
      cur_foreground = win32functions.GetForegroundWindow()

      # if it is already foreground then just return
      if self.handle != cur_foreground:

          # get the thread of the window that is in the foreground
          cur_fore_thread = win32functions.GetWindowThreadProcessId(
              cur_foreground, 0)

          # get the thread of the window that we want to be in the foreground
          control_thread = win32functions.GetWindowThreadProcessId(self, 0)

          # if a different thread owns the active window
          if cur_fore_thread != control_thread:
              # Attach the two threads and set the foreground window
              win32functions.AttachThreadInput(
                  cur_fore_thread, control_thread, True)

              win32functions.SetForegroundWindow(self)

              # detach the thread again
              win32functions.AttachThreadInput(
                  cur_fore_thread, control_thread, False)

          else:   # same threads - just set the foreground window
              win32functions.SetForegroundWindow(self)

          # make sure that we are idle before returning
          win32functions.WaitGuiThreadIdle(self)

          # only sleep if we had to change something!
          time.sleep(.06)

      return self

And here is a short bit of code you can test it with

from pywinauto.application import Application
import time

app = Application.start("Notepad.exe")
app.UntitledNotepad.MenuSelect("Format->Font")

for i in range(4):
  time.sleep(2)
  app.Font.Edit1.SetFocus()

If you run that code and work with other windows - then the Notepad Font dialog will popup (annoyingly :-) ) every 2 seconds.

All is left now is to add a call to this method in those other methods that REQUIRE the window to have focus (TypeKeys() for sure, maybe the ...Input() methods - but they probably require a WindowFromPoint/RealChildWindowFromPoint/ChildWindowFromPoint call first to ensure it's necessary).

(all happy with myself for getting that off the todo list - except it wasn't on the todo list :-D )

Released 0.2.5 prior to Boston-PIG meeting

2006-03-08T01:10:00.000+08:00

I wanted to roll up my various changes before I give my presentation to Boston-Pig meeting this Thursday.

Also I will be traveling next Monday to Europe for work, that might mean that I can do more work on pywinauto - but it also might mean that I can do much less.

Going forward I have more refactoring to do (probably code breaking - but I can leave methods in and deprecate them for a release or two), and I still need to work on the functionality that allows a script to work unchanged on different langauges.

Have fun :-) - and don't worry about asking me questions or letting me know if you are using pywinauto - I really really don't get a lot of mails :-) (It is only since releasing my own package that I notice how little feedback I have given to others in the past!).

Usability, Consistency - RULES!

2006-03-07T17:06:00.000+08:00

I just read an interesting list of "new laws" at http://pfeifferreport.com/trends/trend_userexperience.html (found from http://pyre.third-bit.com/blog/archives/000414.html (found from http://planet.python.org/index.html) .

I find that a number of these resonate strongly with me, and put in words some of what I am trying to achieve with pywinauto.

I should print the list and post it above my desk - then when I go to refactor or add new code I can ask myself if the changes obey the rules :-) (I think 6, 8 and 9 are the ones that will be most useful day to day).

Getting the text of a container control is maybe not that critical for the everyday user of pywinauto - but it should be easy to find if they do need it. Here are some examples of how inconsistent this is at the moment...


  listbox.ItemTexts()[item_index]
  ListView.GetItem(item_index, subitem_index)['text']  # returned item is just a dict
  TreeView.GetItem(0,2,4).Text()
  Header.GetColumnText(col_index)
  Statusbar.GetPartText(part_index) # at least these last two are similar
  Rebar.GetBand(band_index).text # attribute set in returned ctypes structure!

These kinds of inconsistencies do not help anyone learn pywinauto (I hadn't realized it was so bad :-) )

Just as a counter point to above - it's not all so bad.
control.Select() is defined for most classes and behaves (hopefully) more or less as expected.

So - going forward I am going to try and ensure usability and consistency of pywinauto.

The other point is documentation - there IS documentation for pywinauto - but I definitely do not consider myself a tech writer. From reading the rules - it also occurred to me that my documentation is lacking in a major way - it is not easy to find what you are looking for.
I think a new set of pages which describe in a succinct way how to work with controls will help.

E.g.
Operations on controls:

ListBox
     Select
     SetFocus
     GetFocus
ListView
     Select
     IsSelected
     IsChecked
     Check
     UnCheck
     IsFocused
TreeView
     Select
     ...

Even the list by itself helps - in that it would allow you to see what is available more easily then browsing the current documentation.

Wow - lots of stuff to try and remember...

2006-02-25T04:29:00.000+08:00

I guess this little blog will get lost in the flood of PyCon blog stuff - or else everyone will be so busy at PyCon that this entry will stand out (whether that's a good thing or not I am not sure :-)

It sounds like there are one or two people actively using pywinauto - and I am getting some feedback, some bug reports, some suggestions - It's great - but I need to be organized so I don't loose it. So I am going to put some of it here...

First of all there is a Python meetup in Boston on Thursday 9th of March, and if I can get my act together (a Powerpoint presentation and a good demo I guess?) then I will be presenting pywinauto.

So here are the things I want to track

Some applications implement menu's as Toolbars (Internet Explorer for one) and of course MenuSelect() or MenuClick() doesn't work on those. I need to research if there is some style or setting that says a toolbar button will actually popup a menu. If there is I can update the MenuSelect/MenuClick function to use them, but if not they you will have to document to use one of the following methods:
app.dlg.TypeKeys("%FWF") # &File->Ne&w->&Folder
or
# this functionality is not implemented yet - still
# need to change the interface to Toolbar buttons.
app.dlg.Toolbar.Button("File").Click()
app.PopupMenu.MenuSelect("New->Folder")

For now I should just update the documentation (as I feel that will be the final outcome anyway). Especially as the example given was a control with class WTL_CommandBar (which is a different kettle of fish completely!)
I wanted to add my reasoning for creating pywinauto to the documentation and say why I think it is better then a lot of automation tools out there (the idea at least - I hope the implementation doesn't let it down!)
Some way of specifying unique text when looking for a control. For example if you look for a control with the text "Click this button to save as PNG" (maybe there are other buttons "Click this button to save as XXX", etc), then it would be nice if app.dlg.PNG.Click() would work - but it doesn't :-(. The reason is that the matching algorithm I use (using difflib) takes the overall text match - and the match of PNG in the overall text is low.
The reason that I do not just select the best match (original code did do this) is that it matches sometimes when you don't want to. For example if you want to check if a dialog exists for example you could write

if app.dlg.Exists():
# do something because this dialog exists

but if the searching is not strict enough it will find other dialogs and the script will be broken (and it might be hard to fix). Currently the way around is either specify more of the text:
app.dlg.ClickbuttontosaveasPNG.Click()
or use the window_ method...
app.dlg.window_(title_re =".*PNG$").Click()

One idea was possibly to allow something like
app.dlg.PNG_IN_.Click()
or maybe
app.dlg[".*PNG$_RE_"].Click()
where the _IN_ and _RE_ specify that the text should be 'in' or should be tested using a regular expression. I am not sure about either of these as they appear to be a bit 'magic'.
Finish off the Wait* functions. Currently in application.WindowSpecification (which is the class you get if you do app.dlg, or app.dlg.ctrl) there are the following methods:
Exists(self, timeout = exists_timeout))
WaitReady(self, timeout = window_find_timeout, wait_interval = window_retry_interval)
WaitNotEnabled(self, timeout = window_find_timeout, wait_interval = window_retry_interval)
WaitNotVisible(self, timeout = window_find_timeout, wait_interval = window_retry_interval)

These should all have the same signature, and then I need to fill out the rest of the methods ^Wait(Not)?(Enabled|Visible|Ready)$. Though just writing that and thinking of duplicated lines of code - maybe I should just have a Wait() method e.g.
app.dlg.control.Wait(enabled = True, Visible = False)

would return the control being waited for once it is both Enabled and Hidden (unlikely to be very useful :-) or return None on timeout (or should it raise an exception?)
Clear up where to look for appropriate methods. Most methods are in the control wrapper for that particular control (or it's base class(es)) but some (important) methods are in WindowManager. So to be clear the available methods for a particular window are the union of
- methods of WindowSpecification
- methods of the Wrapper for this control
- methods of the base class(s) of the wrapper
Look into pyAA to help getting information from 'difficult' controls (and possibly other uses for it too!)
Then all my various ToDo items:
- Implement 'application data' that will allow the same script to run on many languages
- Implement basic Recording functionality
- Lots more of everything (wrapped controls, tests, examples, documentation, etc)

Test Infected

2006-02-16T09:45:00.000+08:00

I haven't gotten around to reading much of the XP material (well a little online), but with pywinauto I finally started to do some unit tests. Even though my coverage is still quite minimal - I don't have even one test per method yet (and some require more then that!) - I have found that it has helped me to find numerous bugs that I hadn't known about - or wouldn't have known about until I released!

I end up running the tests or at least part of the tests frequently. I am not quite to the point where I am first creating a test for the functionality that I have yet to code.

I now understand why tests make refactoring easy - you change your code until you feel it is refrigerator ready - then you run your tests and make sure that they run like they did before you ripped out that gangrenous rotting piece of code that you smelled last week. :-)

The next thing that I need to do is to add the examples to the tests - that way I will have some 'integration' tests along with all the unit tests. :-)

I feel a release coming soon - and then I think I may need to take it easy for a little while.

It's not only my code!

2006-02-12T10:58:00.000+08:00

I made some progress this evening - but I seem to be doing more refactoring and catching up on unit test implementation then adding much to the code base.

I have a lot of firsts (for me) with pywinauto.

First project I release publicly
First time I have ever written a unit test (yeah I know - bad :-)
First time I have ever had to worry about non functional changes I make to code affecting others (Like changing the interfaces of classes)
First time I have used subversion or used SourceForge for hosting a project
First time I ever thought of blogging (and maybe you are wishing I never DID!)

I'm enjoying the experience - and I think I am learning quite a bit. Here's hoping that my code is useful and I don't waste people's time with either bugs or excessive changes.

Maybe I should think of having the subversion repository hosted somewhere public - so that other's have a chance of contributing and keeping me in line. I think for now there is lots I can do myself, and I am not sure I want to anyway. I am not sure I have enough experience to be able to do it successfully - and with amazing tools like Python and ctypes - there is a lot one person can do :-)

Packages, smackages!

2006-02-10T22:06:00.000+08:00

I had never created a package before working on pywinauto. I just made scripts and if they required other python files that I wrote then I just stuck them in the same directory and imported them from there

import myotherscipt

Then I decided for pywinauto to try and move things around and be better organized - so I created some packages pywinauto.controls and pywinauto.tests.

What feels strange to me is that these packages are not self contained, I don't mind having to rely on class capabilities defined in other packages (e.g. when testing - that the controls passed in have a Rectangle() method) but I don't like having to import pywinauto.win32defines or other modules from pywinauto. It seems to me that packages should be more self contained then that?

Should I just do away with these packages and have everything as a flat structure? Should I re-structure so that the parts of pywinauto that I do have to import are in a package of their own (that does not need anything from pywinauto)?

Well it looks like __path__ is what I need. I guess my pywinauto.controls is not really a package - and I want a directory just for organization. I think pywinauto.tests is a package - as I need to initialize it, and I expect it to always be imported by using import tests (or import pywinauto.tests)

hmm - a little more analysis needed I guess to resolve those dependencies.

New blog - marketing for pywinauto

2006-02-10T21:24:00.000+08:00

Never thought I would start writing a blog - but it seems to be an important marketing medium for open source projects - so I guess it can't hurt too much :-)