Reading the contents of a list within a window

General discussion about the Phantom Automation Language and related tools.

Reading the contents of a list within a window

Postby robsheer on Fri Jul 10, 2009 11:43 pm

Hi,

I'm trying to automate a business process. I would like to know if Phantom can do what I need.

1. I want to create a form that pops up and the user fills the required fields from a paper
document. Then I want to use the fields from this form to automate a **** process
of filling data within an application. First I need to be able to read the contents from within
a window that returns data from a database request. When that data is auto-select by the
macro another row of data will appear beneath it(this is the data I need to be able to read
because it contains the name I need to match and select from a drop down list).

2. Assuming that step 1 above can be done with ANY windows automation tool available, then I need
to be able to read the member names from a 22-member drop-down list and choose the member
that matches the data read in step 1.

Can this be done with the Phantom language or any other method of writing scripts/macros for
automation?

Thank you,
Rob
robsheer
 
Posts: 4
Joined: Fri Jul 10, 2009 11:32 pm

Re: Reading the contents of a list within a window

Postby john on Sat Jul 11, 2009 3:48 pm

Hi Rob-

Phantom should have no problem doing this. The only issue I see is that Phantom cannot, at this time, create user defined forms for input. It can, however, accept input in question-answer form (Phantom asks a question, the user inputs an answer). User defined forms are a feature that is being considered for a future release.

Phantom can generally read data from a window or control using the built-in 'GetText' function:

Code: Select all
# Create a reference to the control with the desired text
window w = MainWin("MyWindow", "MyClass").MainWin("A Field", "Field"); 
# Get the text from the control
string s = w.GetText();
# Display the text (or do whatever is needed with the text)
disp(s);


This should address reading the window contents you describe in step 1. Phantom can also select items in a drop-down menu (if the Window's class is 'ComboBox' or a derivative) by using the 'Select' function. If it is not a ComboBox, you can simulate key strokes using TypeKeys:

Code: Select all
window w = MainWin("MyWindow", "MyClass").MainWin(1, "ComboBox"); 
# Get the number of items in the ComboBox
int Count = w.GetCount();

# Declare the string you want (this can come from your database window read, or wherever)
string s = "test";

# Cycle through the elements to find the one you want
for(int i = 0; i < Count; i++){
  if(w.GetElement(i) == sDesired){
    w.Select(id);
  }
}

# ---- OR ----
window w = MainWin("MyWindow", "MyClass").MainWin(1, "SomeOtherClass"); 
# Simulate a DOWN keystrok on the control to cycle through the items
w.TypeKeys("<DOWN>");
# Depending on what 'SomeOtherClass' is, you may be able to retrieve the selected text, too:
string s = w.GetText();


There are also many other built-in functions. Full documentation is available on-line under the 'Documentation' menu and then under 'Phantom Language Help'. This has some tutorials and many examples of Phantom functions.

I hope this helps and was able to answer your question! If you have any additional questions, please let me know.

-John
john
 
Posts: 27
Joined: Wed Apr 22, 2009 12:32 pm

Re: Reading the contents of a list within a window

Postby Guest on Tue Jul 14, 2009 10:33 am

John,

Thank you so much for all your help! I have limited programming experience, so
I'm not a programmer, but I have programmed some in C, although no GUIs at all.

I have a third-party application that is used in a high-volume department and many of
the tasks are very repetitive and tedious, lots and lots of room for human error. I'm
trying to automate the repetitive tasks that are most time consuming and cause most
of the human error.

I have uploaded an image of the window to best describe the problem and hopefully with
your help I can either prove or disprove that this can be done with the Phantom language.
The screen shot is located at: ****://****.flickr.com/photos/40483161@N08/3719417261/sizes/o/

I have concealed confidential information for security reasons, but kept the main areas required
for programming purposes. When I use the Windows Declarations Recorder, or WinID or Autoit
Window Info, the only information returned is the "Main Application Name - Window Name".
The information doesn't change AT ALL as I move it to the Patient Lookup and Batch Posting
windows(these are the windows I need to read the "text" from) . For Class Name, it always returns
"Transparent Windows Client". We use the application via Citrix so maybe that's what's keeping me
from being able to read the children windows info. Do you know? If this is not a problem, then continue on,
otherwise I can stop now and not waste anymore time cause we have to use Citrix.

OK, so here's what I need to do... I need to be able to read the text in the Patient Lookup window,
specifically, the "Collapsable info under name if name is clicked". I need to read this string then
search for a specific name that will then be used to compare against a list of 22 names in a drop-down
menu. The same is true for the Batch Posting window, I need to be able to read each of the lines until
I find a matching name for the Description column.

Can this be done with the Phantom language or any other way? My programming experience with GUIs is zero, so I could
really use your help. THANK YOU SO MUCH!

Rob
Guest
 

Re: Reading the contents of a list within a window

Postby john on Wed Jul 15, 2009 1:25 am

Hi Rob-

Unfortunately, if the window is in a Citrix terminal, the options are pretty limited. Phantom can only see information about a window if it is on the Desktop directly. I am not aware of an application that would be able to read inside the Citrix terminal (although one may exist). However, I do have a couple of questions:

1: Are you able to select text in the form and manually copy it and paste it? If so, Phantom can 'read' the text by reading it from the Clipboard (using a TypeKeys to either select Copy from the Edit menu or to simulate CTRL+C, and then use GetClipboardText). You can then paste it (either by using TypeKeys so select the menu or to simulate 'SHIFT+INS' or 'CTRL+V').

Code: Select all
# Click at x,y, where the control is
# Alternatively, you could probably use 'TypeKeys("<TAB>")' to tab to the control
MyApp.MouseClick(0, x, y, 0);
# Simulate a copy
MyApp.TypeKeys("<CTRL-C>"); 
# Get the text from the clipboard
string s = GetClipboardText();


2: Likewise, are you able to manually copy and paste from the drop-down list after the name is entered? If so, you can use TypeKeys to navigate the list, then copy the contents, and check it against the 'desired' value.

Code: Select all
# Click at x,y, where the control is
# Alternatively, you could probably use 'TypeKeys("<TAB>")' to tab to the control
MyApp.MouseClick(0, x, y, 0);
# Navigate the list (this would be in a loop)
MyApp.TypeKeys("<DOWN>");
# Simulate a copy
MyApp.TypeKeys("<CTRL+C>"); 
# Get the text from the clipboard
string s = GetClipboardText();
if(s == desired){
  # Do something
}


It unfortunately is much more difficult (using Phantom, anyway) to automate if WinDR cant get the window information. The only option is to use TypeKeys and MouseClick, and by 'copying' and 'pasting' using the clipboard functions. However, usually it can be done with Phantom if copying and pasting can be done. Sorry I dont have advice for any other applications, but I dont know of any that can 'read' information inside a Citrix terminal without copying and pasting.

I hope this helps. If it turns out you can do 'copy and paste', it may indeed be possible. I have automated some pretty complex processes interacting with different terminals by using only TypeKeys, MouseClick, and clipboard functions. If you have any questions, please let me know!

-John
john
 
Posts: 27
Joined: Wed Apr 22, 2009 12:32 pm

Re: Reading the contents of a list within a window

Postby robsheer on Wed Jul 15, 2009 8:44 am

John,

Thank you so much for your help. You're terrific! You've saved me a lot of time. I checked and I cannot copy/paste from any of the windows. The problem is in fact Citrix, I verified it by reading Citrix forums. Citrix creates Transparent Windows Client for the thin-client application. Apparently, I would have to install Spy++ or Windows Declarations Recorder, or any of the other controls-identifying tools on the Citrix server to get the window info, and if I've understood correctly, I would even have to run the automation script on the Citrix server so it can hook/access the windows controls for the application within the Citrix session. Some people have apparently done this successfully to automate. I'm not sure how safe this would be or if the sysadmin would allow it, e.g. what would happen if I accidentally code an infinite loop, I would be scared that it would hang the Citrix server because it's used by 28 offices. I'm running out of ideas. I really wish I could have done this as the current process steps are too tedious, repetitive and time consuming and could easily be automated if I could access the windows controls. Again, thank you so much!!

Rob
robsheer
 
Posts: 4
Joined: Fri Jul 10, 2009 11:32 pm

Re: Reading the contents of a list within a window

Postby robsheer on Wed Jul 15, 2009 12:40 pm

John,

I just noticed that when I move the mouse over the windows/forms that I need to access inside the Citrix application, I do get a unique Handle number(HWND) for each window inside the application. Can I use this HWND handle with Phantom commands to access and read the data/text I need from the open window/forms?

Thanks,
Rob
robsheer
 
Posts: 4
Joined: Fri Jul 10, 2009 11:32 pm

Re: Reading the contents of a list within a window

Postby john on Wed Jul 15, 2009 5:37 pm

Hi Rob-

Yes, indeed, if you have the HWND you can apply it to a window variable using 'SetHandle'. However, the HWND will not be static, and will be a different value with each instance of the application. The trick will be to find the correct control. I would suggest cycling through the controls using GetChild and GetNextSibling until you get to the control you want (the order of controls will not change). Since each control has a unique HWND, this should be possible:

Code: Select all
window MainApp = MainWin("Main Application Name - Window Name", "*");  # Or from a .dec file

# Get the first child
window Child = MainApp.GetChild();
window Desired;
int i = 0;
int iDesired = 10;  # Or whatever position the desired control is in

# Cycle through the children
while(Child){
  i++;
  # If it is the control position you want, save it
  if(i == iDesired){
    Desired = Child;
  }
  Child = Child.GetNextSibling();
}

# Do whatever you want to do
string s = Child.GetText();
disp(s);


Actually, unfortunately, if the control text (Tag) does not change in WinDR and Spy++, GetText() may not work. Citrix may not allow any information to get out other than the visual display. If this is the case, and copy and paste doesnt work, I am afraid I am out of ideas :| . You would have to install the script on the Citrix server (which as you said is likely not allowed). You can do a quick check to see if GetText will work:

Code: Select all
# For now, get the HWND using WinDR or Spy++... Later, if GetText works, you can cycle through
# the children as shown above.  Try starting with a simple control (like a simple edit field).
int i = InputInt("Enter HWND ID: ");

# Set the handle
window w;
w.SetHandle(i);

# Get the contents
string s = w.GetText();

# if s equals the contents of the control, then it works!
disp(s);


I hope this helps... hopefully we can find some way to automate this!

-John
john
 
Posts: 27
Joined: Wed Apr 22, 2009 12:32 pm

Re: Reading the contents of a list within a window

Postby robsheer on Thu Jul 16, 2009 9:29 am

John,

I tried the GetText() example and entered the HWND manually, but all I get is the title for the window, i.e. form, that is currently open inside the app's main window. What I need is to get the data inside this form, it has numerous fields then a large treeview-like box at the bottom which contains the information returned by the database query, each row is then expandable/colllapsable.

I combined your code and added a few disp commands for debugging purposes, and it appears that the GetChild() function is not able to get the child info, it returns nulls, below is the code. Everything works up to that point, the SetActive brings the window to the top using the window handle as it should, but then it just hangs for a few seconds, and returns the error below. It looks like the GetChild is returning a NULL tag/class. Does this mean the phantom language will not able to get the children via Citrix?

Again, thank you for all your help!

int handle = InputInt("Enter HWND ID: ");

# Set the handle
window MainApp;
MainApp.SetHandle(handle);

MainApp.SetActive();
# Get the first child
disp("MainApp returns ");
disp(MainApp);
window Child = MainApp.GetChild();
disp("First Child read");
disp(Child);
window Desired;
int i = 0;
int iDesired = 10; # Or whatever position the desired control is in
#set NextGen active


# Cycle through the children
while(Child){
i++;
# If it is the control position you want, save it
if(i == iDesired){
disp("Child inside IF statement");
disp(Child);
Desired = Child;
}
Child = Child.GetNextSibling();
}

# Do whatever you want to do
disp("Child at the end");
string s = Child.GetText();
disp(s);

OUTPUT:

Enter HWND ID: 4326668
MainApp returns
Handle: 4326668
First Child read
Tag: '' Class: ''
Child at the end
** ERROR: (1) from 'C:\Documents and Settings\Otto\My Documents\Automation Scripts\GetChild.psc' on line 37 - Window '' not found (Class='', Tag=''). (P00100)
robsheer
 
Posts: 4
Joined: Fri Jul 10, 2009 11:32 pm

Re: Reading the contents of a list within a window

Postby john on Thu Jul 16, 2009 12:25 pm

Hi Rob-

Your code looks correct, and it looks like GetChild is indeed returning a NULL window. The reason for the brief hang is because Phantom spends several seconds looking for a window before giving up and giving the P00100 error.

I have two thoughts:

1. For the initial HWND ID, make sure you are entering the ID that WinDR / Spy++ shows when the cursor is on the application's title bar. I just want to make sure that the HWND you are using is for a top-level window and not inadvertently a child window. If SetActive is used on a child window, it will also activate the parent window.

2. If you are able to record the declarations of the window (using WinDR) and save it as a file (one file each with the cursor placed in the title bar, and one with the cursor placed on one of the controls in the Citrix window) and send the files to support@phantomtest.com, I can do some more diagnostics. If it contains proprietary information, you can always use WinDR's editor feature to change the tags and classes (or use a text editor) to remove any proprietary information. The reason I ask for this is sometimes a window hierarchy is not straightforward, particularly in 'terminal' type applications. You can also copy and paste the text in the .dec files here if you want - just make sure you put them in 'code' tags to preserve the tabs:

Code: Select all
[code]
   In Code Tags
[/code]


If its not a strange window hierarchy, and the HWND really was the top-level, I am afraid I may be out of ideas... :|

-John
john
 
Posts: 27
Joined: Wed Apr 22, 2009 12:32 pm


Return to Phantom

Who is online

Users browsing this forum: No registered users and 8 guests

cron