QAInsight.net, QABlog.com, QABlog.net
Brent Strange's thoughts on Software Quality Assurance and technology

 
Friday, September 29, 2006
 
 

How to become a software testing expert

 
 

I took 57 minutest of TV time the other night (skipped Survivor XVII; Jewish midgets vs. Satanic albinos) and instead turned to Google Video to watch How to become a software testing expert by James Bach. I'd never seen James Bach speak before until that night, and I have to say that I really like his style. If you are Software Testing QA Engineer and have a spare 57 minutes I recomend watching this presentation. I did, and I have to say "I'm a software testing expert" and I didn't even realize it! It's very empowering...

You can download the presentation slides off of Jame's site.

 
   
   
   

 
Thursday, September 28, 2006
 
 

Stupid Testing Trick #1: Quickly access your hosts file

 
 


Today marks the day where QAInsight.net is gonna stir the QA pot, put some fuel on the fire, and add a pinch of spice (ground Cayenne). It's getting hot in here, so take off all your clothes, I am getting so hot... uhum well...anyway. To pour some vivation into this parched QA terrain called QAInsight.net I'm going to kick-off a new series dubbed "Stupid Testing Tricks". Each post in the series will be well... a stupid testing trick that is short, uncanny, and helpful. So let's get the stupid stuff started:

Stupid Testing Trick #1: Quickly access your hosts file

When testing in a browser, there is often a need to edit your hosts file, but traversing the relevant, overpopulated Windows directories to "C:\WINDOWS\system32\drivers\etc\hosts" is a royal pain. Ease the pain by creating a browser Favorite/Bookmark that points to your hosts file.

  1. From the browser's Address bar navigate to "C:\WINDOWS\system32\drivers\etc\"  (or wherever your hosts file is located)
  2. Create a Favorite/Bookmark to this location
  3. Right click the Favorite/Bookmark and click "properties"
  4. Change the "Target" field to C:\WINDOWS\system32\drivers\etc\hosts (or wherever your hosts file is located)

When you need to edit the hosts file you just click the bookmark and select an application to open it (you will need to do this since the file can't be permanently assigned to an application due to its lack of an extension).

Have a Stupid Testing Trick? Email me using the "E-Mail" link on the right.

 
   
   
   

 
Monday, September 25, 2006
 
 

SQAG seeking readers

 
 

SQG (Solo QA Guy) seeking readers. QAInsight.net is so lonely...

I guess it would help if I blogged about QA a bit more...

 
   
   
   
 
Sunday, September 24, 2006
 
 

James Bach Exploratory Testing Dynamics outline

 
 

Exploratory Testing Dynamics defined! Do you remember when you thought exploratory testing was just simple software exploration that usually resulted in some pretty cool and unique defects? A chance to test without hindrance of documentation? Get over it; everything must have a rhyme and a reason my QA friend.

 
   
   
   

 
Sunday, September 24, 2006
 
 

Ajax Security Basics and testing

 
 

I've recently had a chance to write some Ajax in a side project that I've been working on and through use of it I started thinking about how one could easily use it to do evil things. Doing evil things reminds me of security testing, and I haven't had an opportunity to test an application that uses Ajax but am pretty interested in finding some good exploits when I do get the chance. Before you get all "You had the chance to test it Brent, didn't you test YOUR Ajax code Brent? You're in Software QA and you don't test your own code?". Let me tell you that I did think about it being exploited, and if it did it wouldn't really matter in my situation. :-)

But while thinking about it, I did find the following article on Ajax Security Basics that would help a tester start thinking about how to attack the technology. After working with it, and reading the article, when I think about how dangerous this could be to an application I rank it up there with the danger of using <frames>. Are any of you testing Ajax applications? Do you have any advice or test cases you'd be willing to share?

 
   
   
   
 
Sunday, September 24, 2006
 
 

2006 CSI/FBI Computer Crime and Security Survey

 
 

Here's an interesting report: 2006 CSI/FBI Computer Crime and Security Survey

 
   
   
   
 
Saturday, September 23, 2006
 
 

More IE 7s added to User Agent Switcher XML file

 
 

I've included the following IE 7 user-agent strings to my importable XML file for User Agent Switcher:

  • MSIE 7 beta (Win XP SP2)
  • MSIE 7 (Win XP SP2)
  • MSIE 7 (Win 2003 Server)
  • MSIE 7 (Win Vista)
  • MSIE 7 (Windows-RSS 1.0)

Download it here.

 
   
   
   

 
Tuesday, September 19, 2006
 
 

NUnit fails with System.IO.FileNotFoundException

 
 

What seemed to be out of the blue, NUnit started failing on me yesterday when I attempted to load my project. The cryptic error was:

System.IO.FileNotFoundException : File or assembly name nunit.core, or one of its dependencies, was not found.

Exception details are found at the bottom of the post. The problem? The web.config that went along with my assembly wasn't valid because I was missing a trailing quote:

<add key="blah" value="missingquote />

Ooops. Thanks for the uninformative error message NUnit. That's a half hour of my life I'll never get back....

System.IO.FileNotFoundException...

Server stack trace:
   at System.Reflection.Assembly.nLoad(AssemblyName fileName, String codeBase, Boolean isStringized, Evidence assemblySecurity, Boolean throwOnFileNotFound, Assembly locationHint, StackCrawlMark& stackMark)
   at System.Reflection.Assembly.InternalLoad(AssemblyName assemblyRef, Boolean stringized, Evidence assemblySecurity, StackCrawlMark& stackMark)
   at System.Reflection.Assembly.InternalLoad(String assemblyString, Evidence assemblySecurity, StackCrawlMark& stackMark)
   at System.Activator.CreateInstance(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityInfo, StackCrawlMark& stackMark)
   at System.Activator.CreateInstance(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityInfo)
   at System.AppDomain.CreateInstance(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at System.Runtime.Remoting.Messaging.StackBuilderSink.PrivateProcessMessage(MethodBase mb, Object[] args, Object server, Int32 methodPtr, Boolean fExecuteInContext, Object[]& outArgs)
   at System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(IMessage msg, Int32 methodPtr, Boolean fExecuteInContext)

Exception rethrown at [0]:
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at System.AppDomain.CreateInstanceAndUnwrap(String assemblyName, String typeName, Boolean ignoreCase, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, Evidence securityAttributes)
   at NUnit.Util.TestDomain.MakeRemoteTestRunner(AppDomain runnerDomain)
   at NUnit.Util.TestDomain.Load(String assemblyFileName, String testFixture)
   at NUnit.Util.TestLoader.LoadTest(String testName)

 
   
   
   

 
Monday, September 18, 2006
 
 

Top 5 Security Vulnerabilities dectected from compromises

 
 

Promoting CISP (Cardholder Information Security Program), Visa has published the educational bulletin: Top Five Data Security Vulnerabilities Identified to Promote Merchant Awareness. To summarize the top 5 vulnerabilities are:

  1. Storage of Track Data
  2. Missing or Outdated Security Patches
  3. Vendor-Supplied Default Settings and Passwords
  4. SQL Injection
  5. Unnecessary and Vulnerable Services on Servers
 
   
   
   

 
Sunday, September 17, 2006
 
 

Inexpensive Web page automation

 
 

On this blog you hear a lot of SWEA this, SWEA that... SWEA solves all my web page automation tasks, SWEA saves my ass, SWEA is cheap, without SWEA I couldn't test 90% of a new build of our UI in 10 minutes, SWEA takes 4 letters out of SWEAt, no SWEAt with SWEA, I get left with T, Time. Time to focus on complex functional tests, installers, performance, (insert more buzz-word test types here).

So about now you're thinking, SWEA Sure Would Enhance the Automation in my workplace. Yeah! YEAH! Now you're catching on...

So, not long ago AdventNet posted a nice little automation tool comparison chart, but they forgot to add SWEA. Wonder why? Probably because SWEA is $59.00 which blows all other tools out of the water in the price department. Not only that, SWEA holds its own in features too. That's pretty freakin' sweet since SWEA was create by one man, a Mr. Alex Furman man. One might think, "Hmm, I'm not sure if I should bank my automation investment on a tool created and supported by one dude". Let me tell you, and you must listen my automation tool shopping friend: Alex Furman is the man! I work with quite a few tools, I've worked with some of the tools that you'll see in the comparison chart (the one I'm leading up to) plus some others, and I have never received the support I've received from Alex. Alex gets shat done, he works hard, he is proud of, and smart about his product. For example, one day I'm like all IMing Alex: "Hey, it'd be nice if I could attach the SWEA designer to any already open Internet Explorer window", Alex is all "Sweet idea, my heroic QA engineer Brent Strange, I'll put that feature in tonight, it'll be a lot more fun than that near-impossible AJAX support I've been working on". Heh. Seriously the guy responds and makes it happen for me (thanks Alex). The tool has made my QA life easier.

Oh yeah, the purpose of this post...If you put it in the comparison list provided by AdventNet it really hangs with the big dogs (view a larger comparison of tools here):

Features

Comparisons

 

SWEA

QEngine WebTest

WinRunner

SilkTest

Rational

Cost / License

$59

$799+

$8000

$6,495 plus $1,170 for Maintenance

$2,900 - $5000

Platform & Browser Support

Platform Support

Windows

Windows & Linux

Windows

Windows & UNIX

Windows

Browser Support

 IE

IE,
Mozilla & FireFox

IE &
Netscape

IE &
Netscape

IE &
Netscape

Recording

Floating menu's

Yes

Yes

No

?

?

Object Name Map

 Yes

Yes

Yes

Yes

Yes

Object Mode Recording in Windows Application

 No (only browser based UI)

No (only browser based UI)

Yes

Yes

Yes

MouseDown Events

 Yes

Yes

Yes

Yes

Yes

Modal / Modaless Dialog

 Yes (Modal)

Yes

?

?

?

Script Language

.Net languages

Jython

TSL

4Test

SQABasic

Extensible

 Yes

Yes

Yes

Yes

Yes

Checkpoint Libraries

?

Yes

Yes

Yes

Yes

PlayBack

Play control
(Fast/Normal Mode)

 Yes

Yes

Yes

?

?

Interoperability

Partial
(Browser)

Full Support
(OS & Browser)

Partial
(Browser)

Partial
(Browser)

Partial
(Browser)

Multi-Window Handling

 Yes

Yes

Limited

Yes

Yes

Automated Regression

 Yes (NUnit)

Yes

Partial

Yes

Yes

Play against different host without re-recording

 Yes

Yes

?

?

?

Error Handling

 Yes

Yes

Yes

Yes

Yes

Validation

Multiple Validation with expression support

 Requires NUnit

Yes

No

No

No

Validation - Window

 Requires NUnit

Yes

Yes

Yes

Yes

Validation - Text

 Requires NUnit

Yes

Yes

Yes

Yes

Validation - Image

 Requires NUnit

Yes

Yes

Yes

Yes

Validation - Table

 Requires NUnit

Yes

Yes

Yes

Yes

Validation - Document

 Requires NUnit

Yes

Yes

Yes

Yes

Validation - Page Response Time & Status

 No

Yes

No

No

?

Validate & Save Window

 Yes

No

Yes

Yes

?

Validate Clipboard Text

 No

No

No

No

?

Validate & Synchronize Screen Rectangle

 No

No

Yes

Yes

?

Validate - DB Data

 Requires NUnit

Yes

Yes

Yes

Yes

Validate & Synchronize Text

?

No

Yes

No

No

 

Data Driven Testing

Yes

Yes

Yes

Yes

Yes

Portability

Yes

Yes

?

?

?

In Built support for Script Maintenance

 No

CVS

No*

No*

No*

Parameterization

?

Yes

Yes

Yes

Yes

* - at extra cost

Others

Customizable

 Yes

Yes

No(separate tool reqd)

Yes

Yes

Internationalization

 Yes

Yes

Yes

Yes

Yes

Detailed Reports

No

Yes

Limited

Yes

Yes

 
 

Application Support

Web, HTML, DHTML, ASP, , JavaScript

Web, HTML, DHTML, ASP, Java, JavaScript

HTML, XML, JavaScript, Java, ActiveX

HTML, XML, JavaScript, Java, ActiveX

HTML, DHTML, Java,
Visual Basic, Visual C++, Oracle Developer
/2000, Delphi, SAP, PeopleSoft and Sybase Powerbuilder.

Applet Testing

 No

No

Yes

Yes

Yes

 

? - Unknown

How do I use SWEA? Check out the tutorial of my test framework with SWEA, NUnit and C#.

 
   
   
   
 
Saturday, September 16, 2006
 
 

Borland SilkTest 2006 is "Available"

 
 

SilkTest 2006 has been released... to customers; and in this release they have one baked feature:

Integration with Borland® StarTeam
SilkTest 2006 supports integration with Borland's StarTeam 2005 that enables you to manage and share your SilkTest project files, test scripts, custom include files, and data files.

The SilkTest source control integration with StarTeam supports file check-in, check-out, get-latest-version, and other functions typical for source control systems.

also included in this release is the unbaked, beta release?? feature:

SilkTest TrueLog
TrueLog is a powerful technology that simplifies root cause analysis of test case failures via visual verification. The results of test runs can be examined in Silk TrueLog Explorer. When an error occurs during a test run, Silk TrueLog Explorer enables you to easily locate the line in your 4Test script that generated the error so that the issue can be resolved.

Silk TrueLog Explorer for SilkTest 2006 is a beta release.

I can see the TrueLog feature being pretty helpful for troubleshooting post run. Read more info here.

SilkTest has a lot of competition now days and in comparison can be a tad spendy at $6500. Hey, did you know that the SWEA and NUnit automation solution will cost you a whopping $59.00? I've used both SilkTest and SWEA for web page testing and I prefer SWEA. Yes, SilkTest is capable of a lot more, but if you're looking for plain ol' web page automation make sure to give SWEA a test drive.

 
   
   
   

 
Wednesday, September 13, 2006
 
 

Update to my User Agent Switcher XML file

 
 

Back in December I blogged about Chris Pederick's FireFox extension User Agent Switcher that does browser user agent spoofing. At the time I provided a small list of user agents that could be imported into the cool little extension. Since then I've modified and added quite a bit to that list. I'm sharing my new user agent list with you here. The new list now includes these browser OS combinations (thanks to Philip Forteza for the Linux versions):

*Internet Explorer*
MSIE 7 Beta  (Win XP)
MSIE 7 Beta  (Win Vista)
MSIE 6 (Win XP)
MSIE 5.5 (Win 2000)
MSIE 5.5 (Win ME)
MSIE 5.0 (Win 95)
MSIE 4.01 (Win 95)
MSIE 4.0 (Win NT)
*FireFox*
FireFox 2.0 Beta2 (XP)
FireFox 1.5.0.6 (Fedora Core 5/Linux)
FireFox 1.5.0.4 (Debian Linux)
FireFox 1.5.0.3 (Mac OSX)
FireFox 1.5.0.1 (Win Vista)
FireFox 1.5 (Win XP)
FireFox 1.4 (Win XP)
FireFox 1.0.7 (Win XP)
FireFox 1.0.6 (Win XP)
FireFox 1.0.5 (SlackWare/Linux)
FireFox 1.0.4 (FreeBSD)
FireFox 1.0 (Win XP)
FireFox 0.9.3 (Windows XP)
*Opera*
Opera 8.0 (Win 2000)
Opera 7.6 (Server 2003)
Opera 7.51 (Win XP)
Opera 7.5 (Win XP)
Opera 7.5 (Win ME)
Opera 4
Opera 3
Opera 2
Opera 1
*Netscape*
Netscape 8.1  (Win XP)
Netscape 8.0.1 (Win XP)
Netscape 7.2 (Win 2000)
Netscape 7.1 (Win 98)
Netscape 4.8 (Win XP)
Netscape 4.76 (Win 98)
Netscape 4.5 (Win 98)
Netscape 4.05 (Win 95)
Netscape 3.01 gold (Win 95)
Netscape 2.02 (Win 95)
*Safari*
Safari 3.0 (v521)
Safari 2.0 (v412)
Safari 2.0 (v401)
Safari 1.3 (v162)
Safari 1.3 (v158)
Safari 1.3 (v146)
Safari 1.2  (v125.8; OSX 10.3.4)
Safari 1.1 (v100.1; OSX 10.3.2)
Safari 1.0 (v85)
Safari 1.0 (v74 beta 2)
Safari 1.0 (v73 beta 2)
Safari 1.0 (v64 beta)
Safari 1.0 (v51 update 1.0 beta)
Safari 1.0  (v48 public beta)
*AOL*
AOL 9.0/MSIE 6.0 (Win XP)
AOL 8.0/MSIE 6.0 (Win XP)
AOL 7.0/MSIE 5.5 (Win 98)
AOL 6.0/MSIE 5.5 (Win 98)
AOL 5.0/MSIE 5.0 (Win 98)
AOL 5.0 (OS X)
AOL 5.0 (Mac PPC)
AOL 4.0/MSIE 3.02 (Win 95)
AOL 4.0/MSIE 4.01 (Win 95)
AOL 4.0 (Mac PPC)
AOL 4.0 (Mac 68k)
AOL 4.0/MSIE 3.0  (Win 3.1)
AOL 3.0/MSIE 3.02 (Win 95)
AOL 3.0/MSIE 4.01 (Win 95)
*Mozilla*
Mozilla 1.7.8 (Linux)
*Spiders/Bots*
Googlebot 2.1 (New version)
Msnbot 1.0
Msnbot 1.0 Beta (v0.11)
Yahoo Slurp
Ask Jeeves/Teoma