AccessGrid.org

Quality Assurance Test Protocol

Introduction to Quality Assurance Testing

Quality Assurance testing aims to improve Access Grid sessions by improving the quality of individual Access Grid nodes. The test includes measurement of a number of metrics ensuring that the service level of an Access Grid node is at an acceptable standard. These metrics cover the categories of audio, video, networking and the use of related applications. QA tests also provide a means to disseminate good practice to node operators. The results of passed QA tests will be made public on this website.

Passing/Failing the QA Test

It is in no one's interest for a node to fail a QA test. The aim of testing is to improve the quality of facilities and operators. Therefore, if simple adjustments can be made immediately in order for a node to pass a metric, this will be allowed. That said, it is preferred that nodes be as prepared as possible to pass the test. If time does not allow for quick troubleshooting or there are numerous problems, then a node will fail the test.

If a node fails the QA test, a retest will be scheduled as soon as possible after the initial test.

As with any test, the secret of success is preparedness. Do not simply "show up" and expect to pass. The standard for a "pass" is intentionally being set high to address the widespread concern about the current (lack of quality) in Access Grid sessions. Much of the criticism leveled at the Access Grid is due to poor configuration of participating nodes. Here are a few ways in which to prepare for your test:

  • Read the test in advance. If there is something you don't know how to do, the time to learn is now - do not wait until the test proper. Go to town hall sessions, local community debug sessions, etc. Unless noted, you must satisfy all required criterion to be granted QA status.

  • Audio is the most critical component of the test. You must ensure that your node's audio is clear, echo-free, and of the proper level. This requires actually listening to your audio from a remote receiver's point of view. Using an Audio reflector service or monitor the audio from a PIG in another room. Sites with poor audio will fail the test - guaranteed.

Structure of the Quality Assurance Test

A QA test is scheduled to last for two hours, but will often take much less time. Components of the QA test fall into four main categories, audio, video, networking and supporting tools. We reserve the right to add or remove metrics and change methods of measurement according to good practice.

Audio

Echo

To pass this category, the QA Tester will speak in a normal speaking voice, the tester will then check to see if they are able to hear their own voice "echoed" back. If no perceptible echo is detected, this will be considered a pass.

Required

Quality

A "QA Tester" will assess the audio quality of the speech coming from the node being tested. The audio quality should be excellent and voices must be clearly understood and there should be no distortion in the voice.

The audio levels of the speakers should be at an acceptable level where in a normal speaking voice, the LED's in RAT should peak at orange, but not go into the red. The speaker will be asked to read something, with some emphasis on parts to check the peaks. The speaking level should also be consistent and not varying (ie, the audio levels continually change from a low level to a loud level) and no overdriven audio should be detected. If a node has these qualities, it will be given a pass for this category.

Required

Ambient

To pass this category, no audio should be detected once an AG’s audio is muted (Either by the microphones or by Rat).

This category will test the ambient room noise. This will be achieved by switching on the microphones, but have no-one speaking. To pass this category, the background ambient noise should not light up more than the first 3 LEDs within the gain meter in RAT and the ambient noise should not be that perceptible nor should it be distracting. (It may be possible for a system to pass if more than 3 led's are currently highlighted if no sound can be noticed)

Required

Video

Picture Quality

To pass this category, the contrast between the background and speakers in the foreground should be good. The presenters image should be clear and not too dark without any image flicker.

 

Required

Camera positioning of Subjects

To pass this category, the cameras should be able to be positioned so that a head and shoulders shot of the presenters can be shown. A camera should also have the ability to display a group of people easily.

Required

Networking

NTP

Each of the AG Nodes will have to be NTP synchronized to pass this category. The node operator will be asked if each of the AG nodes is NTP synchronized and a verbal answer will be acceptable.

Required

Unicast

To pass this category, the AG node must be able to connect to a Unicast Bridge as specified by the regional qa-testers and confirm all audio and video streams through rat and vic.

The following Unicast Bridges are suggestion for the different regions:

Australia - APAG

United States - Argonne

Required

Multicast

To pass this category, the node operator will confirm that they are using multicast and confirm all audio and video streams through rat and vic.

Optional

Beacon

To pass this category, a beacon must be install and displaying on one of the following beacon servers:

UK nodes-http://ulcc.beacon.ja.net/ag/

ASIA-PACIFIC nodes-http://beaconserver.grangenet.net/

Global nodes: http://beacon.dast.nlanr.net/

Please note that the beacon software doesn’t have to install on one of the AccessGrid nodes. As long as the computer the beacon software is running on is a computer on the same network segment as the AccessGrid nodes.

Optional

AGTK

Restart

To pass this category, you must be able to restart the Venue Client and be able to interact with very little adjustment. The main purpose is to ensure that all the services are configured correctly and minimal effort is required to join back into the session. An example of a poorly configured node is if the operator has to adjust the audio gain levels every time the client is restarted.

AGTK Nodes Required

inSORS

Restart

To pass this category, you must be able to restart the inSORS software and be able to interact with very little adjustment. The main purpose is to ensure that all the services are configured correctly and minimal effort is required to join back into the session. An example of a poorly configured node is if the operator has to adjust the audio gain levels every time the client is restarted.

inSORS Nodes Required

Text Messaging

Jabber

To pass this category, you must be able to connect, using the Jabber protocol, to a designated AG3 chat room. Users other than AG3 can simply use any jabber client to connect to the designated chat room.

Required

Shared Applications / Node Services

IGPix

The QA tester will test the use of the IGPix application. To pass, you must be able to connect to an IGPix session.

Optional

UK - Required

Shared Browser

(AGTK2 or AGTK3)

To pass this category, you must be able to run the shared browser application.

Optional

Shared Desktop

(AGTK2 or AGTK3)

The QA tester will test the use of the shared desktop application. To pass, you must be able to connect a shared desktop session.

Optional

Canada - Required

Shared Presentation

(AGTK2 or AGTK3)

To pass this category, you must be able to send and receive a shared presentation.

Optional

AUS - Required

US - Required

TigerBoardAG

(AGTK3 Only)

To pass this category, you must be able to create and join a Tiger Board AG session.

Optional

VPCScreen Capture

(AGTK3 Only)

 

To pass this category, you must be able to transmit and receive a VPCScreen Capture Stream in H261AS mode. To transmit a “VPCScreen Capture” video stream, you will need to add the “VPCScreenProducerService” and set the properties to encode using H261AS.

 

Optional

AUS - Required

VenueVNCClient

(AGTK2 or AGTK3)

The QA tester will test the use of the VenueVNC Client application. To pass, you must simply be able to connect to a Venue VNC session.

Optional

AUS - Required

US - Required

VenueVNCServer

(AGTK2 or AGTK3)

The QA tester will test the use of the VenueVNC Server application. To pass, you must be able create a VenueVNC session and have the ability for external users to be able to connect.

Therefore, it is extremely important any firewalls they may affect connections, to be configured correctly.

Optional

Additional Applications

VNCViewer

The QA tester will test the use of the VNC Viewer application. To pass, you must simply be able to connect to a VNC session.

Optional

VNCServer

The QA tester will test the use of the VNC Server application. To pass, you must be able create a VNC session and have the ability for external users to be able to connect.

 

Therefore, it is extremely important any firewalls they may affect connections, to be configured correctly.

Optional

VNCReflectorViewer

The QA tester will test the use of the VNC Viewer application. To pass, you must simply be able to connect to a reflected VNC session.

Optional

VNCReflectorServer

The QA tester will test the use of the VNC Reflector Server application. To pass, you must be able create a VNCReflecter session and have the ability for external users to be able to connect.

Therefore, it is extremely important any firewalls they may affect connections, to be configured correctly.

Optional

Miscellaneous

Details

The pass this category of the QA Test, each site must have their node registered and all the relevant information is recorded.

Note, the relevant information will be check against the node's information within accessgrid.org nodelisting database.

Required

login or register to post comments