Saturday 26 February 2011

Web Testing Gotchas

There are bugs discovered when testing a Web App that seem to occur regularly. In this Blog I will attempt to keep a log of the ones I and fellow esteemed QAs have discovered.


Technical
  • Back-end servers/services being unavailable or timing-out or getting bad response .. list services called.
  • Character encoding (UTF8-ISO), e.g. £ sign, currencies
  • URL encoding, special chars on GETs and in query strings
  • Invalid HTML ... not valid as per W3C
  • Accessibility not considered (e.g color blindness, alt tags)
  • New images etc not being accepted by client due to browser caching
  • Putting host in links without DNS entry
  • Performance test not accounting for latency ...Web app calling services unnecessarily .. need to check/test using tools like TCPMon
  • Clicking info link not opening in separate tab (window ie6)
  • Changes not implemented on all pages (basket, checkout) plus visa-versa
  • Submit / bookmark after logged-out so should need to re-authenticate and redirect
  • Locale set to non GB and so may affect currency time etc
  • Handling zero items
  • Client under load causes odd sequencing
  • Tab/filter selections not remembered
  • Query string exposing too much
  • Different envs (e.g. not localhost) behaving differently for signin/out
  • Updates should be POSTs and not GETs to avoid caching issues
  • Data coming back from service is missing
  • Print document pagination
  • Selecting an option un-selects another option on the same page
  • Common elements not appearing (exist and order) consistently on different pages
  • Using out-of-date versions of back-end services
  • tomcat icon being left on the app tab
  • just enter the domain without the app in the url
  • tomcat apps also being deployed to live (root, manager)
  • missing http to https redirect
  • test artifacts also being deployed in wars and so being accessible
  • A change on the page which is not consistent with other stuff on the page
  • Browser holding cache of static content which should be cleared before test
  • Incorrect caching in proxy server
  • Redirects being cached
  • able to delete/put get only resources
  • double query parameters e.g. referrer
  • Templates having hard-coded values
  • Data created before cut-over can be handled by new app


Penetration


  • Editing html  directly e.g. buy stuff that you can't normally or changing hidden status
  • XSS and Sql Injection
  • Url re-writing used to access back-ends .. should be signed



Cross Browser

  • IE7 not handling apostrophe from json
  • Cross browser, particularly CSS images and Javascript behaviour in IE6
  • IE6 not having tabs
  • Forgot about IE8 compatability mode
  • IE8 not supporting SVG
  • adding 0 to start of number makes it octal on Chrome/FF

Cookies 
  • Not being picked-up .. need to check what is in the cookie
  • Edit manually .. cookie not being signed and obfuscated
  • Session cookie when should be permanent or perm cookie when should be session
  • Not expiring so anyone can get access
  • Personal info in cookies being un-encrypted
  • Unexpected cookies
  • Manually cleared mid flow
  • Disabled in brwoser
  • Sign-out clearing cookies and direct url navigation to bookmark not picking-up empty cookie rather than no cookie
  • Cookie (outside of app) domain not matching app
  • Not being cleared when Browser shuts down
  • Cookie hijacking .... someone copying your cookie

Behavioural
  • Proceed to next page without making a valid selectionn
  • Using Back button to view personal info on public sites
  • Tick > 1 box when boxes should be mutually exclusive
  • Back/forward button and do click
  • Multiple browser tabs
  • Navigate to pages directly using URLs and trying an action * check navigate to other pages bypassing current or navigating to this page bypassing dependent pages
  • F5 / refresh losing info
  • Bookmark / deep-link pages
  • Screen resolution causing rendering issues
  • Checking the second link in a list
  • Entering resource and double slash at end
  • Not checking all functionality on page 2 of pagination
  • Illogical filtering, e.g. for a product that does not exist on page 2
  • 2 concurrent sessions on same account on different browsers
  • integration with other systems (up-line and down-line)
  • going to temporary / status pages via url
  • resend same request e.g. buy twice
  • re-setting filter action resets data but not filter drop-down
  • a change to a field that exists on more than one page
  • double-click a button having undesired effect
  • Can navigate to file on web app 
  • Repeating action twice e.g. sign-out twice
  • Forgot to test app wide stories such as sign-out from anywhere
  • Sign-in/out not taking account of deep-link
  • Deeplink not taking account of portal authentication
  • Flats or business addresses or house names in address search
  • Entering invalid password too many time
  • Clicking enter instead of submit
  • Address entered that has no delivery
  • Drop-down list not having a default or the default not being 'need to select'
  • Sorting OK by higher level but random at lower level
  • Case sensitivity email address or post code

Javascript
  • JS errors hidden/swallowed
  • Disabled Javascript
  • JS is not obfuscated
  • Fast click, particularly when JS and Ajax are running, causing race condition




State Transitions
  • State transitions going not being picked up on all scenarios
  • Go through full journey twice 
  • Click confirmation twice 
  • Dependent elements on a page not interacting e.g. filter and reset filter
  • States that are related but can both bet set independently e.g. order confirmed and order expired
  • Review content of cookie and model regularly
  • End of process changing state but app has cached previous state


Data entry
  • Rounding / precision
  • Case sensitive
  • List of dates overlapping end of year
  • Session timeout
  • Client time vs Servertime, which should it be in logs and DB
  • Multiple errors on same page
  • Missing about / help / info
  • Empty entry
  • Open form but ignore and submit
  • Leading, embedded and trailing spaces
  • 3 decimal place currencies
  • Number: 0, -ve, decimal, integer, non number
  • Dates: past, future, leap
  • Special html chars (e.g. <, &, £)
  • Size: min / max size - inconsistent with persistence
  • External file data is in mixed case

WebDriver / Selenium II
  • Need to maximise window
  • Scroll element into view
  • Xpath case sensitive in IE
  • Clicking elements where the click trigger is not activated
  • Not handling redirects properly
  • Screen moves after checking so location is different
  • Interact with hidden elements .. can do with JS
  • HtmlUnit performing a GET when using the 'back button' functionality

General
  • New system has missed something that exists in old system
  • Not pre-planning every Story
  • Not having list of states and edge cases and journeys
  • Not holding a bug-bash prior to release
  • Not having a state * action matrix
  • Not referrring to this list