Instructions: Do you buy psychological tests on the World Wide Web? Do you use tests purchased on the Web in cost-consequential applications (i.e. recruiting, selection or development)? If so, consider the following standards of professionalism and competence.For each category, select the description that best (not necessarily perfectly) describes the testing organization you currently purchase from, or one you may be in the process of evaluating.Try to give an answer in each category. If you don't know or aren't sure of your answer, use your best judgment. When you've completed this checklist, click on the "Evaluate" button to estimate the "fitness" of your cyber-testing organization.

Close This Window

Location
Company operates out of a physical location with a street address, accepts credit cards, and otherwise conducts business according to legitimate practices
 
 Company has only a box number, Internet address or other non-traditional workspace in lieu of physical premises
 
Education
Test creator formally educated in psychological test construction in a traditional residential university degree program
 
 Test creator places bloated emphasis on questionable credentials; actual training, if any, may be peripheral to some other non-psychological degree program (a "doctor" of chiropractic or professional minister, for example); or credentials obtained via distance learning, correspondence school or online institutions granting junk degrees for a fee.
 
Experience
Company has verifiable and relevant history in the market
 
 Company makes claims but provides no easy way to actually verify past history; may offer pre-packaged excuses for what past employers or clients might say (Note: Some former associates may be unwilling to risk legal problems by giving candid references. In such cases, look for the absence of clear positives)
 
References
Company has appropriate credit and business references readily available for inspection on request
 
 Company makes rhetorical, unsubstantiated claims of success, citing unnamed "Fortune 500 clients" and "international clients;" individuals claim to be "in demand as a keynote speaker" without providing details
 
Documentation
Company is willing to submit technical manuals and other appropriate documentation, "warts and all," for indepndent evaluation
 
 Company provides only desktop printouts/photocopies, delays or refuses to provide documentation
 
APA Compliance
Company provides APA (American Psychological Association, or local equivalent) compliant or nearly compliant test manual and all required supporting statistical information
 
 Company provides only anecdotal testimonials from unqualified end users or fragments from isolated, unpublished studies; may distort, downplay or be unaware of APA compliance issues
 
Geneaology
Company supports claims with a substantive, cumulative body of research evidence
 
 Supporting research is spotty, limited in scope and history, or consists of student research papers
 
Appropriate Conduct
Company is professional; respects the intellectual efforts and assets of competitors; appropriately cites sources used and acknowledges copyrights and trademarks of others
 
 Company takes sole credit for work done by others; cites few references or untraceable ones, e.g., unidentified sales managers or "shills"
 
Currency of Practices
Company's technical studies conform to conventional, modern practices such as supplying statistical "main effects," "power analysis" and others that indicate familiarity with current procedures
 
 Company provides only simplistic correlations or context-isolated statistical significance tests (Note: Statistical significance does not mean a research result is proven or even practically useful!)
 
Theoretical Foundation
Company's tests are derived from an explicit theoretical basis
 
 Company's tests are based on untested and sometimes incoherent musings about personal experience recklessly generalized to everybody else
 
Whole-Body Research
Company's tests have undergone systematic verification by many studies, in various settings, under different conditions
 
 Company's tests have been subject to only a handful of recent, uniformly self-serving studies
 
Believability
Company presents studies that both confirm and refute the use of the test, acknowledging that no test is perfect and all possess inherent limitations
 
 Company makes hyper-sensational broadbrush claims of applicability, stating or implying that the test works everywhere, for everybody (Note: properly constructed tests may achieve 30% predictive accuracy under optimum conditions; beware of claims of 80%, 90%, or higher accuracy)
 
Legal Defensibility
Company can and will attest that its test is an original work, lawfully owned and created and not now or in the past subject to legal dispute over its origins or content
 
 Company resists attempts to verify its outright ownership of the test in writing, or conceals the fact that the test is, has been, or may be legally contested in the future
 
Originality
Company's test is essentially original, unique or pioneering Company's test is the product of a testing "chop shop" - the work of others, rearranged, repackaged and given a different name; test represents a cut-and-paste adaptation of information known to have appeared elsewhere first (and perhaps not suitable for its currently marketed purpose)

 
 
 

Close This Window