GUI Testing的檢查清單


很多QA只會做functional testing, 除了這些之外, 其他種類的測試便不知道要如何進行. 尤其在GUI方面, 因為缺乏資料, 即使有心想做的人也不知道要如何進行. 再加上無人重視, 更不容易會將這項工作納入處理.

這篇是我在網路上, 找到對GUi的檢查清單. 我想這是一份很好的入門指引, 讓你快速瞭解到, 專業的 GUI應該要注意到哪些事情. 你可以把這份清單當做起始點, 然後來建立你自己的知識庫.

Enjoy it!!

GUI Checklist
http://jaanujeeva.blogspot.com/2008/12/gui-checklist.html

1. USER INTERFACE
    1.1 COLORS
        1.1.1 Are hyperlink colors standard?
        1.1.2 Are the field backgrounds the correct color?
        1.1.3 Are the field prompts the correct color?
        1.1.4 Are the screen and field colors adjusted correctly for non-editable mode?
        1.1.5 Does the site use (approximately) standard link colors?
        1.1.6 Are all the buttons are in standard format and size?
        1.1.7 Is the general screen background the correct color?
        1.1.8 Is the page background (color) distraction free?

    1.2 CONTENT
        1.2.1 All fonts to be the same
        1.2.2 Are all the screen prompts specified in the correct screen font?
        1.2.3 Does content remain if you need to go back to a previous page, or if you move forward to another new page?
        1.2.4 Is all text properly aligned?
        1.2.5 Is the text in all fields specified in the correct screen font?
        1.2.6 Is all the heading are left aligned
        1.2.7 Does the first letter of the second word appears in lowercase? Eg:

    1.3 IMAGES
        1.3.1 Are all graphics properly aligned?
        1.3.2 Are graphics being used the most efficient use of file size?
        1.3.3 Are graphics optimized for quick downloads?
        1.3.4 Assure that command buttons are all of similar size and shape, and same font & font size.
        1.3.5 Banner style & size & display exact same as existing windows
        1.3.6 Does text wrap properly around pictures/graphics?
        1.3.7 Is it visually consistent even without graphics?

   1.4 INSTRUCTIONS
        1.4.1 Is all the error message text spelt correctly on this screen?
        1.4.2 Is all the micro-help text(i.e tool tip) spelt correctly on this screen?
        1.4.3 Microhelp text(i.e tool tip) for every enabled field & button
        1.4.4 Progress messages on load of tabbed(active screens) screens
    
    1.5 NAVIGATION
        1.5.1 Are all disabled fields avoided in the TAB sequence?
        1.5.2 Are all read-only fields avoided in the TAB sequence?
        1.5.3 Can all screens accessible via buttons on this screen be accessed correctly?
        1.5.4 Does a scrollbar appear if required?
        1.5.5 Does the Tab Order specified on the screen go in sequence from Top Left to bottom right? This is the default unless otherwise specified.
        1.5.6 Is there a link to home on every single page?
        1.5.7 On open of tab focus will be on first editable field
        1.5.8 When an error message occurs does the focus return to the field in error when the user cancels it?
    
    1.6 USABILITY
        1.6.1 Are all the field prompts spelt correctly?
        1.6.2 Are fonts too large or too small to read?
        1.6.3 Are names in command button & option box names are not abbreviations.
        1.6.4 Assure that option boxes, option buttons, and command buttons are logically grouped together in clearly demarcated areas “Group Box”
        1.6.5 Can the typical user run the system without frustration?
        1.6.6 Do pages print legibly without cutting off text?
        1.6.7 Does the site convey a clear sense of its intended audience?
        1.6.8 Does the site have a consistent, clearly recognizable “look-&-feel”?
        1.6.9 Does User cab Login Member Area with both UserName/Email ID ?
        1.6.10 Does the site look good on 640 x 480, 600×800 etc.?
        1.6.11 Does the system provide or facilitate customer service? i.e. responsive, helpful, accurate?
        1.6.12 Is all terminology understandable for all of the site’s intended users?
Performance & Security Testing Checklist

2. PERFORMANCE
    2.1 LOAD
        2.1.1 Many users requesting a certain page at the same time or using the site simultaneously
        2.1.2 Increase the number of users and keep the data constant
        2.1.3 Does the home page load quickly? within 8 seconds
        2.1.4 Is load time appropriate to content, even on a slow dial-in connection?
        2.1.5 Can the site sustain long periods of usage by multiple users?
        2.1.6 Can the site sustain long periods of continuous usage by 1 user?
        2.1.7 Is page loading performance acceptable over modems of different speeds?
        2.1.8 Does the system meet its goals for response time, throughput, and availability?
        2.1.9 Have you defined standards for response time (i.e. all screens should paint within 10 seconds)?
        2.1.10 Does the system operate in the same way across different computer and network configurations, platforms and environments, with different mixes of other applications?

    2.2 VOLUME
        2.2.1 Increase the data by having constant users
        2.2.2 Will the site allow for large orders without locking out inventory if the transaction is invalid?
        2.2.3 Can the site sustain large transactions without crashing?

    2.3 STRESS
        2.3.1 Increase both number of users and the data
        2.3.2 Performance of memory, CPU, file handling etc.
        2.3.3 Error in software, hardware, memory errors (leakage, overwrite or pointers)
        2.3.4 Is the application or certain features going to be used only during certain periods of time or will it be used continuously 24 hours a day 7 days a week? Test that the application is able to perform during those conditions. Will downtime be allowed or is that out of the question?
        2.3.5 Verify that the application is able to meet the requirements and does not run out of memory or disk space.

    2.4 SECURITY
        2.4.1 Is confidentiality/user privacy protected?
        2.4.2 Does the site prompt for user name and password?
        2.4.3 Are there Digital Certificates, both at server and client?
        2.4.4 Have you verified where encryption begins and ends?
        2.4.5 Are concurrent log-ons permitted?
        2.4.6 Does the application include time-outs due to inactivity?
        2.4.7 Is bookmarking disabled on secure pages?
        2.4.8 Does the key/lock display on status bar for insecure/secure pages?
        2.4.9 Is Right Click, View, Source disabled?
        2.4.10 Are you prevented from doing direct searches by editing content in the URL?
        2.4.11 If using Digital Certificates, test the browser Cache by enrolling for the Certificate and completing all of the required security information. After completing the application and installation of the certificate.

kojenchieh 發表在 痞客邦 留言(0) 人氣()

撰寫好的Bug Report之技巧與秘訣

How to write a good bug report? Tips and Tricks
http://jaanujeeva.blogspot.com/2009/01/how-to-write-good-bug-report-tips-and.html

最近老闆發現某個product, 有20%~30%的bug被reject. 中間的原因可能很多, 但是也突顯了bug report 的重要性. 如果你能產生有效的bug report, 能準確提供問題所在之處, 這樣才能讓所有人都達到win win的地步.

這裡找到一篇bug report 的經驗談, 大家享用吧!!

1. 為什麼要產生好的bug report
- 如果你的bug report能有效率的指出問題所在, RD越有機會能更解決它
- Cem Kaner.曾說“The point of writing problem report(bug report) is to get bugs fixed”
- 如果QA沒有辦法產生出正確的bug report, RD最後有可能說這bug是沒有問題的, 因而reject它. 這將會導致QA的士氣其受損, 甚至防衛心理或對立心理而升高


2. 好的Bug Report應該具備有怎樣的特質呢?
(1) Having clearly specified bug number:
- Always assign a unique number to each bug report.
- This will help to identify the bug record.
- Note the number and brief description of each bug you reported.

(2) Reproducible:
- If your bug is not reproducible it will never get fixed.
- You should clearly mention the steps to reproduce the bug.
- Do not assume or skip any reproducing step.
- Step by step described bug problem is easy to reproduce and fix.

(3) Be Specific:
- Do not write a essay about the problem.
- Be Specific and to the point.
- Try to summarize the problem in minimum words yet in effective way.
- Do not combine multiple problems even they seem to be similar.
- Write different reports for each problem.

一些能幫助你寫好bug report的小技巧
(1) Report the problem immediately:
- If you found any bug while testing, do not wait to write detail bug report later. Instead write the bug report immediately.
- This will ensure a good and reproducible bug report.
-If you decide to write the bug report later on then chances are high to miss the important steps in your report.

(2) Reproduce the bug three times before writing bug report:
- Your bug should be reproducible.
- Make sure your steps are robust enough to reproduce the bug without any ambiguity.
- If your bug is not reproducible every time you can still file a bug mentioning the periodic nature of the bug.

(3) Test the same bug occurrence on other similar module:
- Sometimes developer use same code for different similar modules.
- So chances are high that bug in one module can occur in other similar modules as well.
- Even you can try to find more severe version of the bug you found.

(4) Write a good bug summary:
- Bug summary will help developers to quickly analyze the bug nature.
- Poor quality report will unnecessarily increase the development and testing time.
- Communicate well through your bug report summary. Keep in mind bug summary is used as a reference to search the bug in bug inventory.

(5) Read bug report before hitting Submit button:
- Read all sentences, wording, steps used in bug report.
- See if any sentence is creating ambiguity that can lead to misinterpretation.
- Misleading words or sentences should be avoided in order to have a clear bug report.

(6) Do not use Abusive language:
- It’s nice that you did a good work and found a bug but do not use this credit for criticizing developer or to attack any individual.
- Conclusion: No doubt that your bug report should be a high quality document.
- Focus on writing good bug reports, spend some time on this task because this is main communication point between tester, developer and manager.
- Mangers should make aware to their team that writing a good bug report is primary responsibility of any tester.
- Your efforts towards writing good bug report will not only save company resources but also create a good relationship between you and developers.

kojenchieh 發表在 痞客邦 留言(0) 人氣()

Verification v.s. Validation

上週在幫engineer 上training課時, 問了他們什麼是verfication 和validation, 很多人不知道他們是什麼, 以及有什麼區別

這裡是我簡單找到的定義, 希望能幫助你區分兩者的差別

Verification:
- The set of activities that ensure that software correctly implements a specific function
- Are we building the product right?
- Ensure that the product has been built according to the requirements and design specifications.

Validation
- A different set of activities that ensure that software that has been built is traceable to customer requiremetns
- Are we building the right product?
- Ensure that the product actually meets the user's needs, and that the specifications were correct in the first place



kojenchieh 發表在 痞客邦 留言(0) 人氣()

Agile Requirements Collaboration (1)

Beyond Story Cards: Agile Requirements Collaboration
http://jamesshore.com/Presentations/Beyond%20Story%20Cards.html

James Shore是"The Art of Agile Development"一書的作者, 在agile是赫赫有名的作者. 他在2005年受邀到Software Process Improvement Network (SPIN)演講. 主題是Agile Requirements Collaboration.

在這演講中, James主要討論的是以下的主題
- what happens before you create the story cards and
- what happens after you create them
但是他並不提story本身的細節


1. 為什麼要 Collaborate?
(1) 這裡他引用"Project Management: The Criteria for Success"的內容, 讓我們知道正確需求的重要性:
Lack of user involvement traditionally has been the No. 1 reason for project failure.
Conversely, it has been the leading contributor to project success.
Even when delivered on time and on budget, a project can fail it it doesn't meet user needs or expectations.
(2) 所以他認為:   
user involvement is a key success factor in software projects... in all software projects, agile or not.
(3) Agile development values collaboration because it increases your chance of success.
- 所以作者要調強的是要如何ehance collaboration,來進一步釐清需求. 而不是只有把它寫出來, 貼在牆壁上而已

2. 那stories從哪裡來呢?
(1) On an XP project, that person is usually called the "on-site customer."
- You'll also hear terms like "lead customer" and "product manager."  
- I'm going to call him (or her) a "product manager" in this presentation, because it's more in line with common industry terms.
(2) The product manager needs to have a strong, clear vision of the product
    - but that doesn't mean he or she is solely responsible for it.
    - work with interaction designers, business analysts, real customers, marketers, users...
    - has to have a strong sense of vision, to tie all the diverse threads together into a strong, usable product.
    - has to know the market well and understand what it needs.
    - has to have the political savvy to juggle all of the project stakeholders' demands, and he has to have the authority to make his decisions about priorities stick.
(3) 所以作者覺得XP其中一個最難的地方, 是找到好的product manager

=========================================================
觀眾所提的問題

1. Audience question: What if you don't have anyone with a vision of the product?
Ans:

(1) I haven't encountered that.  What I have seen is that people sometimes aren't sure what they want.
(2) They think they know, but when they actually see it running, they realize that they wanted something else.
(3) I think the frequent iteration-by-iteration deployment of agile methods are an important way to help people understand on what they really want, because they get to see the software actually running.
(4) 這裡有作者以前的工作經驗:
- I was on a team where we were working with a customer and we were showing him the product every two weeks.
- We weren't sure that we were really delivering what he wanted.
- Every time we said, "this is really what we're going to deploy!" And he said, "sure, sure."
- And then, about four weeks from release, we said that again and he said, "wait! That's what we're going to deliver? No, no, it's all wrong."
- But it only took us one two-week iteration to change the software to what he wanted. Most of the changes were just user interface changes.

2. Audience question: I think something that's more common is to have too many people with product vision. How do you balance this?
Ans:

(1) That's the product manager's role, to balance the competing visions of all the different stakeholders. You need someone with political savvy to do that.
(2) I've tried a number of different options, such as allocating each group a portion of the iteration budget, having people justify the value of their stories, and they didn't work well.
(3) We spent too much time doing bits and pieces of too many things and never got anything completely done. Ultimately what worked best was just having a product manager make the call.

=========================================================

The "latest responsible moment"

(1) Story card hell
- Story card hell is when you have 300 story cards and you have to keep track of them all. This doesn't work.
- Story cards aren't meant to capture all of the details of your product requirements. They're only supposed to be a sentence or two. At most, a paragraph.
- 作者舉了一個例子
    * You have one story card that says "make toast" and another that says "cook bread."
    * Is that the same story? Or are they different requirements for completely different parts of the system?
    * You can't tell, and you lose track of the overall vision because you're surrounded by trees.

(2) 發生"Story card hell"的徵兆
- A good sign that you're in story card hell is when you start hearing the question, "can we put these in a database?"
- Every customer I've ever worked with wanted to put story cards in a database. That misses the point of story cards.
- If you're worried about losing story cards, you're doing too much with them.

(3) Latest Responsible Moment
- "Latest responsible moment" is a core agile idea.
- The term comes from Lean Software Development.
- The idea is that you want to put off decisions as long as you can: to the latest responsible moment.
- But it's the latest responsible moment, not the "last possible" moment. That wouldn't be responsible.

(4) 為何要用"Latest Responsible Moment"
- The reason we do this is because putting off decisions is cheaper and results in better decisions.
- Money we spend tomorrow is cheaper than money we spend today, and between now and tomorrow, we'll learn more things about our environment and the decisions we need to make.
- Plus, things could change so much that the decision isn't relevant any more.
- If we've spent the time and effort to make the decision, that money is wasted.
- So we wait as long as we responsibly can.

kojenchieh 發表在 痞客邦 留言(0) 人氣()

如何進行Sprint Planning (下)

How To Implement Scrum In 10 Easy Steps - Step #4: Sprint Planning (Tasks)
http://www.agile-software-development.com/2007/10/how-to-implement-scrum-in-10-easy-steps_11.html

前面在sprint planning meeting中, 已經釐清好backlog的reuqirement, 接下來就是規劃要完成這些reuqirement的細部工作項目, 以及所需的時間.


1. 和前面釐清需求會議的關係
(1) Although Part 2 of the workshop can follow straight on from the first part, it is sometimes helpful for there to be a short gap between the two meetings; maybe 1 day.
(2) This allows time to clarify any outstanding questions arising from part 1 of the workshop before proceeding with the next step.


2. 有哪些角色需要參加工作項目規劃的會議
    - Business Analysts if you have them. Testers if you have them.
    - ALL Developers on the Scrum team for the product.
    - The Product Owner and any customer, user or business representatives need not attend this part (part 2) of the Sprint Planning workshop, as it’s likely to be more technical in nature and is more about the team working out how the selected backlog items will be delivered.
    - However, they should be welcome to attend if they wish, which may help their understanding of what’s involved to deliver the features, and may help if any further clarification is required as the tasks are discussed and estimated.


3. 設定你有多少時間可以花在這個Sprint
(1) Calculate the team’s Sprint Budget: the available number of hours the team has to work on the Sprint.
    (The available hours in the Sprint Duration X  the number of full-time people in the Sprint )
    + (For people who are working part-time in the Sprint, include the number of hours they can commit to)
    - (Any reasonable deductions for time that team members will not be able to spend working on the Sprint)
(2) Deduct holidays, any known meetings, any time likely to be spent working on other projects, etc.
(3) Based on past experience, deduct a reasonable amount of time for support, if appropriate.
(4) Make sure all these calculations are transparent and visible to all.


4. 把需求展開成真正處理的工作項目
(1) Go through each Product Backlog item selected for the Sprint. Break the requirements into tasks.
(2) Tasks may include the traditional steps in a development lifecycle
    - For instance: Design, Development, Unit Testing, System Testing, UAT (User Acceptance Testing), Documentation, etc.
(3) Remember, agile software development methods do not exclude these steps. Agile methods just advocate doing the steps feature-by-feature, just in time, instead of in big phases.
(4) Each of these tasks, especially development, may be broken down further.
(5) Include all tasks necessary to make the Product Backlog item 100% complete
(6) Agree as a team on your definition of done, so everyone is aware what will have to be completed and included in the estimates.
(7) State tasks as deliverables, if at all possible.
    - Deliverables are more measurable than tasks.
    - Instead of describing what you’re going to do, describe what you’re going to deliver.


5. 以小時為單位來評估你的工作項目
(1) Keep tasks small. Estimate all tasks in hours. Estimate each task as a team.
(2) Ask everyone what they think, in order to identify missed tasks, or to identify simpler solutions.
(3) Ideally task estimates should be no more than 1 day.
(4) If an estimate is much larger than this, the requirements should be broken down further so the tasks are smaller.
(5) Keeping tasks small enough to estimate at less than 1 day has some specific benefits.
    - Breaking tasks down into very small chunks means they are easier to estimate. The accuracy of your estimating will be improved as a result.
    - Tasks less than 1 day are more measurable in the daily Scrum (stand-up meeting). 1 day tasks are either done or they are not.


6. 承諾哪些backlog要在這個sprint中完成
(1) Add up all the task estimates for the selected Product Backlog.
(2) If they are significantly over the team’s Sprint Budget, reduce the number of Product Backlog items selected for the Sprint.
(3) Remember the Product Backlog was in priority order, so if possible it should be the lower item(s) on the backlog that are removed from the Sprint.
(4) The remaining list of estimated Tasks – those tasks needed to complete the selected Product Backlog within the Sprint - is your Sprint Backlog.
(5) The team should commit to delivering the Sprint Backlog.


7. 找出後續要作的工作
(1) Sometimes teams under-commit or over-estimate. Stranger things have happened! :-)
(2) In my experience this is quite common when teams are new to Scrum.
    - This can sometimes result in an over-cautious approach to the estimates.
I think it’s because they are unfamiliar with the process and potentially out of their comfort zone initially. They may not have had much experience of estimating in the past.
And they may not have been asked to commit to their  own delivery before.
This can sometimes result in an over-cautious(謹慎的) approach to the estimates.
(3) Always include some additional scope in your Sprint Backlog, over and above what you think can be achieved.
    - This is important in order to have something ready if the team delivers early, as the Sprint should ideally remain a fixed length.
    - Clearly identify these items as Stretch Tasks.
(4) Should never expect Stretch Tasks to be reached.
    - No-one should ever be beaten up if Stretch Tasks are never reached.
    - And if you do manage to complete any Stretch Tasks, this should be cause for celebration!



kojenchieh 發表在 痞客邦 留言(0) 人氣()

Close

您尚未登入,將以訪客身份留言。亦可以上方服務帳號登入留言

請輸入暱稱 ( 最多顯示 6 個中文字元 )

請輸入標題 ( 最多顯示 9 個中文字元 )

請輸入內容 ( 最多 140 個中文字元 )

reload

請輸入左方認證碼:

看不懂,換張圖

請輸入驗證碼