Ian Bicking: the old part of his blog

Twill in Javascript

Comments:

Hmmm... My opinion on this mixed. Competition is a good thing, but at the same I can see lots of opportunity for collaboration. With so many testing tools around, it would be fun to do some kind of "Testing Tools Summit" to share ideas and compare notes. Maybe host it in conjuction with a "Python Web Frameworks Summit". :-)

Do you think Selenium is just too complicated to get going or do you think it's just lacking documentation? What are your top 3 three things you'd like to see (or see fixed or see changed) in Selenium, that weren't there?

-Jason "The guy who wrote Selenium" Huggins.

# Jason Huggins

---

I don't have any real problem with Selenium. In part I did this because it amused me, but also the ability to do automated/non-browser evaluation of the script is important. Like I said, it wouldn't be that hard to write a non-browser Selenium interpreter (maybe that even exists), but I just wrote it in this direction.

# Ian Bicking

---

Well, as long as I'm on the topic, this is what I like less about Selenium:

• There's a lot of up-front complexity that isn't very important, and little things (like how do you start your own test suite) are not as blindingly obvious as they should be.
• Last time I looked at it -- and this was in discussion, so it may have been resolved -- there wasn't many ways to create higher level abstractions of your scripts, to keep the line count down and cut out repetetive parts. (Note that the featureset provided by my code is, if anything, worse on that regard)
• The names are really long.
• The table is a bit too constrained. Everything must take 0-2 arguments.
• Interactive development seems like it would be hard -- I am not likely to type Selenium commands out easily. Interactive creation of Twill in the browser seems quite reasonable (not implemented, of course).

But all that said, none of those were the reasons I wrote this, nor are they the kind of substantive reasons I would or wouldn't choose Selenium.

# Ian Bicking

---

Yeah, I do think the HTML table syntax can be limiting sometimes. I'm starting to prefer Twill's syntax of splitting on whitespace, instead of table tags. If you write your tests in Mozilla Composer or Word or Excel, Selenium's tables thing is not a big deal. I know some people also use PHP or JSP to convert code into the table tags on the fly. Since I spend my whole day in TextPad on Windows, I think I'd rather have a clean syntax that looks good in plain text. I'll blame Selenium's original design on the fact that it started as a JavaScript implementation of the FIT framework, which relies heavily on HTML tables for its test syntax.

I think both Twill and Selenium are little too terse in its syntax. Instead of this (twill syntax):

# Not too exciting...
go /examples/twill_js/test.html
fv 0 age 10
fv 0 name Ian
echo Get ready for the submit!
submit 0
notfind NOTHING
follow back
code 200

I'd rather have a HyperTalk-inspired syntax like the following:

-- Not too exciting
go to /examples/twill_js/test.html
type 10 into field with name "age"
type "Ian" into field with name "name"
print "Get ready for the submit"
click "submit-me"
assert page is not null
go back
assert response_code is 200

I think the HyperTalk-esque syntax has the same "non-programmer friendliness" that Selenium table tests or Twill tests have, but in a more literate and self-documentating syntax. However, there are now decades of experience with English-like languages like AppleScript, HyperTalk, (and even COBOL), and most "real programmers" hate those languages, even though in this case, I think it would make sense-- a domain specific language for automating actions on a web page. Most programmers would rather have Ruby, Perl, JavaScript, Python, or Java, etc. But anyway, that's a whole 'nother debate, and I don't think I have time to (re)invent the HyperTalk programming language to improve the readibility of my Selenium tests, even though I'd love to do it. I throw this idea out there just in case someone else in web-land stumbles upon this thread and has a free year to implement it for me, and agrees that HyperTalk was the coolest language ever. :-)

With that tangent aside, I'm thinking that the logical next steps for Selenium are:

1. To make the HTML table-tag aspect of Selenium tests optional. Split on whitespace instead, like Twill.
2. More fully support straight JavaScript test files, for the moments when you need loops, conditionals, functions, etc...
3. More fully embracing browser extensions like Selenium Recorder... Browser extensions let Selenium run "outside" of the security sandbox Selenium usually lives in. And being "outside" that box would let users do interesting things like testing or automating any site on the net (not just the the local host where Selenium was installed.) The key words for this are "cross site scripting" and "same origin policy"... and it trips up lots of new users of Selenium.

Since Selenium "lives in the browser", you could do (or hopefully soon do) interactive inspection and test creation by using cool tools like JavaScript Shell (http://www.squarefree.com/shell/) to play with a page and create tests. Also, interesting browser extensions like Selenium Recorder (http://seleniumrecorder.mozdev.org/) are making it even easier to get up and running with Selenium and creating your first tests...

Anyway, 'tis interesting times to be a web developer and software tester. Lots of cool stuff is getting created these days in this space.

-Jason

# Jason Huggins

---

Hi, Jason,

thanks for your positive comments on twill's syntax: I make it up as I go, and (thus far) have mostly been just happy when something works. I do try to keep the syntax simple as opposed to easy, because one of the major use cases for twill is its use by non-programmers; what programmers find easy is often not so simple. Note also that the twill syntax is very closely linked to Python code; for example,:

go here

translates directly to:

go('here')

This is a very convenient feature for Python programmers and several users seem to be using it this way.

On a more general note, I'm a bit boggled by Ian's ability to work on 80 billion different projects. At least for the short term, I don't have the capacity to work on a JavaScript version of twill; nor am I sure that it'd be an unambiguously good thing to have, unless it were somehow possible to keep the languages virtually identical. Maybe it is, maybe not. However, it may be that Selenium and Twill can get together and have unnatural spawn in the future: perhaps a twill-inspired version of Selenium with pwettier syntax?

Also, let me say: twill aims low. I want a simple, solid way to script and test Web sites in a fairly black-box-ish manner. It honestly shouldn't be too much work to get there, and I bet the future will be in recorders and/or test automation technology rather than twill-ish languages. After all, who cares what language the test uses, if the computer can write it for you?

Finally -- Grig Gheorghiu and I have proposed an Agile Development tutorial for PyCon. As you no doubt know, Grig is a huge proponent of Selenium. We will certainly feature both twill and Selenium in our tutorial. No doubt our febrile imaginations will come up with plans for some orcish melange of twill and Selenium, and I have a hard time resisting a good hack...

--titus

p.s. "--" for comments is ungodly! long live '#'!

p.p.s. just kidding ;).

# Titus Brown

---

I bet the future will be in recorders and/or test automation technology rather than twill-ish languages. After all, who cares what language the test uses, if the computer can write it for you?

I disagree, I think recordings can never understand what matters and what doesn't, and the result is too fragile. And once you've done it, if it's not expressed as some at least vaguely sensible language, then it is uneditable. That language might be in markup, or might not, but I think it really has to be a "language", not just a literal recording. Recorders are nice, still, they just aren't complete by themselves.

# Ian Bicking

---

"Recorders are nice, still, they just aren't complete by themselves."

I agree. Test recorders are kind of like of Ruby on Rails's generators and scaffolding... They both get knocked alot, however, they are very useful when you consider them "the starting point" and not the final production-ready result. Test recorders lower the barrier to entry into the testing framework's language or API--- that, and the kids love 'em. :-) My past experience with macro or test recording is with SQA Robot by Rational (now IBM) and with Microsoft Excel's built in macro utility. In both cases, I used recording to generate a simple, single working script... then went on to edit that raw test into a far more robust and "programmed, not recorded" test suite. The recorders gave me working code from step one, which I then manipulated as needed.

# Jason Huggins

---

Well, maybe it's a matter of degree. At some point I doubt there will be a need for language innovation -- but recorders can get much better...

We'll see ;).

--titus

# Titus Brown

---

SystemMessage: SEVERE/4 (:4)

Missing matching underline for section title overline.

---------------------------------------------------------------------------
Brian Lloyd has developed zope.testrecorder, a cross-browser (IE,
Firefox, Safari) JavaScript app that records browser events (clicking,

# sureshvv

---

Quoting Benji York from gmane.comp.web.zope.devel:

Brian Lloyd has developed zope.testrecorder, a cross-browser (IE, Firefox, Safari) JavaScript app that records browser events (clicking, entering text, etc.) and test assertions (this text box should contain this text, this checkbox should be unchecked, etc.) and renders the captured events as either an HTML Selenium test or a doctest using zope.testbrowser. It can be installed as a product into Zope 2 or as a Python package in Zope 3.

The rendering framework is pluggable and it's pretty easy to add new rendering options for other testing systems (twill, etc.).

The testrecorder is currently in "beta" form, but is very handy, so if anyone wants to use/market/improve/add tests for it, go ahead. :) Get it via Subversion at svn://svn.zope.org/repos/main/zope.testrecorder/trunk

It is very cool.

# sureshvv

---