Testing forms and how anal do we need to be?

🗄️ Archive
1

Recently I have taken a hard look at how I test certain elements and asked myself, am I testing these to the best of my ability or am I testing these with a lack of knowledge; Now I am wondering, have I gone too far?

I am primarily talking about forms and the various input fields. I have taken the stance of entering lots of different kinds of text and characters from alphanumeric, English, Arabic, Japanese characters to emoji’s. The last was laughed at by my developers, but I argued, that a lot of people out there like to be unique and could add an emoji for fun. If I don’t test this scenario and our system falls over because of it, then I would consider myself a failure in that area of testing. So, I still stand fast to my decision on using emojis as a valid test, but to the chagrin of some of my developers. (Yes, they can be immature).

What are people’s thoughts on this subject?

1 Like
2

It depends.

If your application has a defined set of allowed input values for a field, then it should not attempt to save anything that isn’t included in the whitelist. If it’s going to be used by everyone and their dog and needs to take almost any kind of input, then it should be able to save almost any kind of input and only reject input that’s in the defined set of non-valid inputs.

For instance, if your input field is limited to positive decimal values between 0 and 100 inclusive, it shouldn’t allow entry of anything other than digits and zero or one instance of the locale-dependent decimal separator (i.e. 15.55 would be acceptable in the USA, 15.5. would not, nor would 15,55 (but 15,55 would be acceptable in locales that use comma as the separator).) It shouldn’t matter what you try to put in there.

On the other hand, if your input field allows almost anything, you should be able to put emojis in there, or Arabic characters, or kanji, or hieroglyphics, and the application should display what you put in correctly.

If there is nothing saying what a field should take, it can be… interesting… to do things to it that fail logic or sanity tests. Things like the entire text of Hamlet, or negative ages, or completely the wrong data type if you’re allowed to do it. My argument for these tests has always been that if you allow a user to do something, sooner or later a user will do that thing.

1 Like
3

This is an excellent example of why context defines the value of the testing you do.

Some interesting questions to gather context to answer “have I gone too far” are:

  • What language could our users use?
  • What character sets do we permit?
  • What’s the impact of this thing crashing? What even is this thing?
  • Where does the data come from?
  • What restrictions does storing and processing this data tell us about what we can permit as an input?
  • How much do I care? What am I being paid, what’s the boss like, am I annoying people and does that matter?
  • Will anyone else care? If my findings don’t inform decisions then how should I proceed?
  • What are the moral implications of not caring? How about the quality ones?
  • Is there something else that I should be doing that’s more important?
  • Why are the coders taking this attitude to my testing? Are they upset that I’ve worked to a higher quality standard than they chose to implement, probably for good reason? Should I lower my quality standard or educate them on why mine is higher by using compelling stories about failure impact? Or are they just pleased with my thoroughness?
  • Am I tired?
  • Are the coders known for checking over their own work?
  • What does our development methodology look like?

It’s an interesting thought experiment to come up with more questions*. Because testing is infinite and resources are not infinite “good enough” testing is worthy of examining carefully. So much of what we do becomes about cost-effective strategy and logistics, not just great techniques.

*(and why the answer matters)

2 Likes
4

Most of our input fields allow pretty much anything! I’ve tried to argue that we need to lock them down to some basics, but the idea always gets boo’d.

5

If they take pretty much anything, that’s a good time to start thinking about whether they also do SQL or script injection: there’s nothing quite like the discovery that you can enter <script>alert(123)</script> into a text field and the alert pops up after you save.

1 Like
6

“No user would ever do that!” really means “No user that I’ve thought of, and that I like, would do that on purpose."

http://www.developsense.com/blog/2007/07/no-user-would-ever-do-that/

Also, be careful assuming that the users you think of always have good intentions.

2 Likes
7

The biggest problem of Web 2.0 is, any user and every user can try arbitrary input as the applications allow through forms or any other input fields. And adding a “emoji” is a test to me which is meaningful. Maybe the developers can think of 100 ways that a user will not include an emoji inside the form text fields, but developer shouldn’t forget about the combinations a brain can generate based on a situation / context, and also the emotions inside the brain which can make user do anything. Following is an example that I can think of,

A house owner is safeguarding his house with CCTV, well-trained dogs, sensors and what not. Even after having all these security measures, there is still a chance that a thief can get into the house with his / her skills. Also, there is a chance that a thief may be scared. You see that we have both the combination and both are valid ones. So is an “emoji”.

I repeat, any user and every user has the freedom of arbitrary input and if application lacks the validation mechanisms or doesn’t sanitize the inputs, the stakeholders just need to rely on their luck and prayer to the god (And the prayer is to not make any user think about such inputs). And we are doing a serious business here and we don’t rely on prayers and luck when we deal with software development.

Here are some of the high-level ideas that I can talk about,
// Sanitize every input through the text field
// Maintain alert feature to administrator if users try to use some inputs that can pose possible threats (For instance: if someone uses , stop them from using it by HTML encoding. Once stopped, also alert the administrator about this so that administrator is aware of this and can take necessary measures and also sets high-alert).
// I would use “Web Developer” add-on to quickly learn about the form values including the HTML tag attributes (You can download the “Web Developer” addon on Firefox at Web Developer – Get this Extension for 🦊 Firefox (en-US)) You could quickly use Forms utility in Web Developer addon to display the “Form Details”.

I will be looking at maxlength attributes, enabled / disabled values, autocomplete values if on / off, form POST request value and more details in order to test them and question their existence.

// I love Fuzzing and love to use Fuzzer from OWASP (You can read more about Fuzzing at Fuzzing | OWASP Foundation and also there are tools listed and frameworks listed in the end of the article on the webpage)

// Now, fuzzing is not really sufficient. I would love to create a script that exercises across all the characters that exist in the computer world. I would use “Character Map” as my source to input characters in the input fields / form and record the results (request / response if its a website / web application).

// I would look for HTTP headers of forms. For instance if there is caching. What happens if a user makes some entry in the input fields which is invalid, but clicks on the submit form. The request is sent to the server, but server responds with an error for that input field (invalid data). But, the user is surprised to see that all the data in other input fields is also deleted or wiped off (Now, this can be very irritating and is a caching issue).

// Also, I would try to learn more about the code being used to create these input fields / form. Is there any third-party code being reused? Or else there is some JS framework being used without questioning it for its quality? And some more questions (I cannot think of now as I am sleepy :D).

These are some things that I can think of as of now. This is very smaller set of ideas for me and is just an example. Every application poses a different challenge and my ideas are exploratory when I start testing them or start creating a test strategy to test them. I hope this example (high-level) helps!

Cheers!