How I test Games

If you want a career as a Tester, this is for You

I don't test computer games, well, I do, but I'm not paid to. I will hopefully never work in a game software house, but it's not a bad way to start at all. I decided testing games was not for me after a interview at Sony in London. 

In a gray part of London in 2006 (much of London is gray looking once the heat of summer kicks in mind-you), I went to a round building, full of kids. Don't get me wrong, I've been to the Jagex (Runescape) HQ too and those kids are having the same kind of fun, but the pay. So yeah, not for the pay and the hours at all, but I do test games. I started writing games reviews over 3 years ago. and that has attracted attention from small indies who want to get positive reviews written around the time of their launches. Pushing the socials kind of stuff, so they often send out free pre-release copies. When I get a copy I write a review, but I also give defect feedback for any that are in beta, and here is gets tricky.

Games are easy to test and a good start for learning for to file defect reports.

How to write a bug report without any requirements

Well, you do have 2 requirements, the game has to be "playable" and has to make someone money eventually. 

Requirement #3 You have the game description, so when reporting an issue with the game, you want to see where the game description deviates from what your experience is. If the game is aimed at children, it has to use suitable language. Not too cutesy, because kids know when they are being spoken down to, but not too adulty, or else mum won't let her 7 year old play the game. So the smallest bugs you can raise are for things like grammar and spelling, and believe me, these are really easy to screenshot and get corrected. Easy bugs also help you get the developers attention. build relationship, ask them questions about what they hope the game will become or do. And be genuine but kind and build connection. In a real job, bugs are not just a thing you write up once and forget, they are a conversation than can go back and forth. In the industry we call it "bug tennis", where an issue goes back to you because you left out something or explained it poorly. This costs time.

What

Where/When

Why

1. Think small, but think big.

The best time to write a well supported bug for a computer game, is when the code is still very changeable. It's amazing how badly architected games code can be and how hard to change it often can be. Games are usually monolithic and not component-based at all. They are supposed to do one thing, and not have components that make any code changes easy to do in the way applications might be modified. So any suggestions that require a big change are usually far more work than you think.

Above is a screenshot of a game on the left with my suggested "accessibility" change on the right as a mocked up image, to make the text pop more. There are other issues in this screenshot, and my version on the right is not perfect, but it's a 5 minute defect. Always describe what you "expected" things to end up in your bug report. Not what you wanted to happen.

2. Smol QOL changes, bigger wins.

QOL (Quality Of Life) are one step bigger than the smaller (SMOL #slang) cosmetic bugs like fixing spelling, fixing the colour of something or the positioning of something. I see a lot of games get bad reviews not based on the enjoyability of the game, but on it's playability. For example any interaction menu like when clicking on something or responding to an event the user has to perform a repetitive mouse gesture or series of clicks. These break immersion because they waste time. Gamers call these QOL issues.

3. Classification

When writing a bug report, don't write like people do on the forums, come up with a classification system that makes sense for the product, and use the class of defect in the defect title. This will grab a developers attention when they are fishing for defects to fix in the backlog tool whatever that might be. good defect titles make you not only write a good bug report, but also mean you bug will get seen if at all in the flood of emails a developer might see.

I prefer to spend a minute and write a one liner bug title before I start writing up the bug itself. Bug titles should fit in one line in your screen. Example:

"Cosmetic: When catching a butterfly the camera briefly goes fuzzy"

Now this bug might not be a simple one to fix in the end, but it's not a critical bug like a crash or other data loss bug. Hangs, Crashes, Data loss, all of these are what we sometimes call showstoppers. which brings me to another point about bug reports.

Having a good bug title/description will also force you to write the bug report about one item and one issue only. Never lump 2 bugs into one unless they are tightly related.

4. Priority versus risk

Do not ever try to give a bug priority, that is the product owner's job. Lots of the bugs you report will never be fixed, often because the issue you found does not really impact the core mission. Or worse is a hard to reproduce defect, or just a defect that does not do much damage to the end quality. People who post on forums have a habit of making their complaint about a game or an app, the most important. They forget that 90% of people playing the game will not ever notice the thing that they do. Yes, most bugs, tend to only impact a very small number of users. The ones that impact all users are ones that tend to get fixed. Given the choice of 2 defects I might find, I'm always going to raise the one that will impact more. So the tester's job is to describe the impact or risk of a bug impacting a user, not it's priority in the work queue.

Finally: Screenshots Screenshots Screenshots

Before you hit send on your beautiful bug description make sure you have evidence, and are laying out your bug report with the 3 key elements. What, where/when and Why.

I love testing all kinds of software, and hope to improve my written communication, so expect to see more of these. Let me know in the comments are you a paid games tester, I really would love to hear from you.

A different community

Last night I met up with a different community, an obscure underground movement that challenges the educational values held by the captains of democracy. They call themselves happy hackers.
Technology has become more affordable to the mass-market, but at the same time more accessible. You can perform scientific observations, investigations and home learning now, more than ever before. It's a bit like open-source, but for education.

For more on what they do, visit my e-journal.
For that that find this prospect scary, do not visit my techno blog/journal.

Software download statistics

My download report for the last 2 weeks, looks impressive, but what does it mean?
Report for - Modbus - PLC Simulator






Report for - DIP Switch






I will have to come back in a month's time and compare.
Statistics are gathered from these 2 URLs on a 3rd-party download site called BrotherSoft. (nope, not related to Broederbond)
I need to turn this all into a marketing window.

Software Engineering

I once was given a theoretical challenge, write a software engine, that can write software to interface with other system on it's own through learning. A kind of AI but not AI, instead at the end it must write it's own new program. Yes, it is possible if you had enough time for a process to connect up to another process, and try through brute force to start sharing data; finally saving the result as a new program. It's conceivable only because computers are good at repetitive tasks, so long as a human starts both ends off with a fighting chance and there are some rules in place, it should work?

Well the problem was a bit more complicated than that, but I nonetheless went off to consider this problem. Firstly with fear because if it could be done, software developers would start putting themselves out of work, but eventually once convinced it could not be done, I relaxed a little. Well this was a long time ago, perhaps a few decades hence I could re-formulate the same original question properly and on paper (I won't now because that would loose IP), and ask it again.

Engineering is full of questions and solutions that just do not quite work at the time. How many new technologies enabling their completion do there need to be?

Programmer for Hire

• Cambridge, UK - experienced software engineer
• Available immediately
• For details, see my online CV

Job Wanted

Posted my VC online Web 2.0 style this evening.

It's amazing what you can do when you suddenly have time on your hands.

Programming STDERR

It's frustrating when you want to get something to work, and nobody's solution fits. I had already gotten quite far on a problem and had it almost done when I found this neat solution to the problem of piping error messages sent to STDERR to a file on MSDN.

I needed to either write a huge perl-script(and without a debugger that's a cramp) or reidirect the stderr and stdout from a static code analysis tool called splint so that I can gather statistics on any dodgy lines of 'C' code.
Why a beer? Well that's a celebration of a slight change in the wind. Where it will go is probably nowhere, but it's a change nonetheless.

/edit
Returning to the redirecting of standard error and pipes, the downside it the default pipe-buffer size, in fact although you can easily increase the pipe buffer size, we all know a huge buffer is in-efficient. You see, unlesss you can empty a smaller buffer in the background while the child-process is filling it, the child will get blocked if it tries to overflow the buffer. A very bad thing, normally solved by spawning a thread to process and empty things while you wait for the child to terminate. I hate and love threads, so forgoing a complex solution, I solved this by simply waiting 100ms for the child to terminate, and then emptying the stream buffer in between by polling using WaitForProcess(). It appears to solve the problem, but exactly how?.... because someone has to close the stream in order to detect the end and not get blocked yourself in the parent waiting for a read on a dead stream?
As it so happens, the parent can happily close it's copy of the redirected stream handle and then keep reading from the duplicate knowing all will unblock when the child terminates (because the handle closes in the child when it does so). Problem solved.

IBM or ISM?

What happened to ISM? Well todays rant is about an IBM product, called Clearcase.

http://www-01.ibm.com/support/docview.wss?uid=swg1IC50214

The problem is when the client is used disconnected. That's fine you say, but the client insists on doing pop-ups from a Visual Studio plug-in. It's a over simplified diagnostic, and in-ellegant to say the least.

---------------------------
ClearCase
---------------------------
Error determining type of current view.


---------------------------
OK
---------------------------

...I am just annoyed that there is so little help to be found to help users work around this problem, because it is possible if someone just put a tiny bit of effort into the plug-in to add a checkbox into the code, it's about 10 lines of code, but unless more folk stick out their tongues....

How long is a M$ minute?

Once again the Desktop clean-up wizard appears, it seems to wake up every 2 months, and then has the cheek to say that a shortcut that I last used 4 months ago, has never been used.

It seems the M$ definition of 'never' is totally off the wall, and today I am disabling the silly program. I see it has been striped from Vista, I can only guess that the OS is either showing it's age or we are forgetting that how to go about making a system easier to use for beginners, yet powerful in the hands of the advanced user is actually not do-able without a "I am a noob" switch. Am I just blind to all things PC?

/edit
If you look under your desktop properties, there is a tickbox hidden away there. just clear it to stop the wizard.

LAMP or Socket?

Been playing with an Ubuntu distro lately to start setting up a LAMP to host the www.plcsimulator.org domain where I keep my modbus protocol simulator program. My current gripe is that Linux is not overly difficult to learn, is nice and cheap, but just takes time. This is just much more fun than work at the moment, so I am rushing home in the evening to find all the FAQ and a HOW-TOs that all the apache dummies like me can follow. Once I can write some PHP to talk to the SQL-DB, I will effectively have some live website content to publish and maybe have a live "TODO" page that shows people what bugs are being reported, fixed and what features are being added.

A great fan of the self-learnt-man, this exercise generally takes time.
Now all I need to work out is how to get this thing to print out edible choc eclaires.

LUA lightweight embeddable script engine

My compilled release DLL 248K (un-optimised) tells it all, compiler, and runtime all there.
Getting started was not easy, if not frustrating, for such a simple task. in the end, the main resource is www.lua.org, and the LUA wiki, and LUA forums. Amongst the best intro articles I could find: http://www.gamedev.net/reference/articles/article1932.asp - and http://www.debian-administration.org/articles/264, and of course the LUA Wiki. Lua is surprising in 2 aspects. Its' free, and easy to get going once you have the background filled in for you. I suggest digging into the problem, and then just trolling through the reference manual and the LUA online book edition. A pitty ordering a paper copy of the book does not help in a big way, because you only need help getting LUA going for about the first 3 days, pretty much the same time as it takes to deliver if the book was local.
LUA is extensible, and if you have not gotten to this bit in just a day or so, you missed something along the way, other engines take far longer to get extending working in, without any blow-by-blow or click-by-click instructions! something you will not get with LUA.

Why LUA? Well I have decided to get cracking on writting a game, and the biggest issue we could face is not just user-content (modding), but game-desighn. I think keeping the desighn viscosity a bit in-the-air with scripting as glue might allow a direction/desighn change to just 'happen' when needed. The game is to be based on a top-down isometric real-time strategy, with wide unit-upgrade characteristics, which create multiple strategic possibilities. Micromanagement, expansion or defence? which will it be? What part will LUA play, well it's too early to tell, although I can see tonnes of oppourtunity: configuration, AI, 'bulding automation' amongst others.

UNIT testing

White box, Black box, Component test; is your UUT realy stand-alone. Is the interface stand-alone? Most likely not. Is the interface automatable? ...sometimes the input or output from software that interfaces with hardware like a fingerprint scanner might actually require that you need real fingers to get good input data because a testing-layer that allows 'fake' input to get injected was never desighned in. I think that for a test engineer, the big headaches are around, have a specified and desighned enough tests, and should not be around things like. "Is it automatable, can I get it stand-alone and in the spotlight on it's own?". Reality is that integration testing will show you which unit tests you should have created, but never thought of. I think these are the ones where I want to kick myself, because a small failure has caused project hiccups.
I imagine there is a good book on this stuff, (unit test specification that is) and how you can go about nailing the beast down to produce quality in the right places early and easily. I must say I look forward to the next Symbian component testing project (probably some blue-tooth work) ,and I have learned a lot about unit-testing in 6 months. I look forward to Bluetooth, because it is something I know little about, and the possibilities there are really wide for some fun to be had.